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12th  ANNUAL  SYSTEMS  ENGINEERING  CONFERENCE 

“Achieving  Acquisition  Excellence  Via  Effective  Systems  Engineering  ” 

San  Diego,  CA 
26  -  29  October  2009 


Agenda 


Tuesday,  October  27,  2009 

PLENARY  SESSION  1  -  INTRODUCTION  &  OPENING  REMARKS 

•  Mr.  Bob  Rassa,  Director,  Systems  Supportability,  Raytheon;  Chair,  Systems  Engineering  Division,  NDIA 

KEYNOTE  ADDRESS 

•  Honorable  Zachary  J.  Lemnios,  Director,  Defense  Research  and  Engineering,  Office  of  the  Secretary  of  Defense  (Acquisition,  Technology  and 
Logistics) 

PLENARY  SESSION  2  -  SERVICE  ACQUISITION  EXECUTIVES 

VIEW  FROM  THE  TOP:  HOW  CAN  SE  SUPPORT  PROGRAM  EXECUTION? 

Moderator:  Mr.  Terry  Jaggers,  Principal  Deputy,  Systems  Engineering,  Office  of  the  Director,  Defense  Research  and  Engineering 

•  Mr.  David  G.  Ahem,  Director,  Portfolio  Systems  Acquisition,  Office  of  the  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 

•  Mr.  Thomas  E.  Mullins,  Deputy  Assistant  Secretary  for  Plans,  Programs  and  Resources  (SAAL-ZR),  Office  of  the  Assistant  Secretary  of  the 
Army  (Acquisition,  Logistics  and  Technology) 

•  Mr.  Christopher  A.  Miller,  PEO  for  Command,  Control,  Communications,  Computers  and  Intelligence  (C4I),  ASNRDA 

•  Mr.  Randall  G.  Walden,  Director,  Information  Dominance  Programs,  Office  of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition),  AFRCO 

LUNCH  WITH  SPEAKER  IN  THE  REGATTA  PAVILION 

•  Mr.  Stephen  Welby,  Director,  Systems  Engineering,  Office  of  the  Director,  Defense  Research  and  Engineering 

PLENARY  SESSION  3  -  TEST  &  EVALUATION  EXECUTIVES 

VIEW  FROM  THE  TOP:  HOW  SE  CAN  SUPPORT  DEVELOPMENTAL  TEST  AND  EVALUATION? 

Moderator:  Mr.  Jim  O’Bryon,  The  O’Bryon  Group;  Chair,  Test  and  Evaluation  Division 

•  Dr.  James  N.  Streilein,  Technical  Advisor,  HQ  Army  Test  &  Evaluation  Command 

•  Ms.  Amy  Markowich,  Deputy  DoN  T&E  Executive 

•  Colonel  Dexter  M.  Sapinoso,  USAF,  Chief  of  Air  Force  Test  and  Evaluation  Policy  and  Programs 

•  Mr.  Christopher  DiPetto,  OU SD( AT &L)/DDR&E/DT &E 

PLENARY  SESSION  4  -  SE  AND  ACQUISITIONS  REFORM:  THE  WAY  AHEAD 

Moderator:  Mrs.  Kristen  Baldwin,  Office  of  the  Director,  Defense  Research  and  Engineering 

•  Mr.  Ross  Guckert,  Office  of  the  Secretary  of  the  Army  for  Acquisition,  Logistics  and  Technology  (ASA(ALT)) 

•  Mr.  Carl  Siel,  Office  of  the  Assistant  Secretary  of  the  Navy  for  Research,  Development  and  Acquisition  (ASN(RDA)CHSENG) 

•  Colonel  Shawn  Shanley,  USAF,  Chief  Systems  Engineer,  Office  of  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition,  Science, 
Technology,  and  Engineering  (SAF/AQR) 

•  Mr.  Nicholas  Torelli,  Office  of  the  Director,  Defense  Research  and  Engineering 


WEDNESDAY.  OCTOBER  28.  CONCURRENT  SESSIONS 
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TRACK  1 

Systems  Engineering  Effectiveness  -  Bayview  III 

•  885 1  -  Rapid  Development  and  Integration  of  Remote  Weapon  Systems  to  Meet  Operational  Requirements ,  Mr.  Joseph  Burkart,  NSWC  Crane, 
Small  Arms  Air  Platform  Integration 

•  8893  -  Rapid  Development,  Mr.  Michael  Gaydar,  NAVAIR 

•  8847  -  Tailoring  the  SE  Process  to  Effectively  Complement  the  SW  Agile  Development  Process ,  Mr.  William  Lyders,  AS  SETT  Inc. 

•  8902  -  Systems  Engineering  Leading  Indicators:  Insight  into  Effective  Systems  Engineering ,  Mr.  Gary  Roedler,  Lockheed  Martin  Corporation 

•  9414  -  Correcting  Deficiencies  in  the  Systems  Engineering  of  Tactical  Weapons ,  Mr.  Marvin  Ebbert,  Raytheon  Missile  Systems 

•  8948  -  Value  Engineering  Applications  in  Service  Contracts ,  Dr.  Jay  Mandelbaum,  Value  Engineering  Applications  in  Service  Contracts 

•  88 16  -  Mind  the  GAPs-a  Systems  Engineering  Implementation  of  DoDI  5000. 02,  Dr.  Thomas  Christian,  U.  S.  Air  Force 

•  8990  -  Systems  Engineering  for  Rapid  Capability  Development ,  Mr.  Thomas  McDermott,  Georgia  Tech  Research  Institute 

•  8974  -  Transforming  Systems  and  Software  Engineering  Across  an  Enterprise,  Mr.  Jeffery  Wilcox,  Lockheed  Martin  Corporation 

•  8863  -  Using  Requirements  Compliance  to  Identify  Gaps  Between  the  Technical  Solution  and  Requirements,  Mr.  Frank  Salvatore,  High 
Performance  Technologies,  Inc. 

•  8823  -  Win  and  Influence  Design  Engineers — Change  Their  Affordability  DNA,  Mr.  Tim  Morrill,  Raytheon  Company 

TRACK  2 

Early  System  Engineering  -  Bayview  II 

•  895 1  -  USAF  View  ofNRC  “Pre-A  Systems  Engineering ”  Study  Committee  Recommendations  As  Addressed  By  Levin-McCain  (P.L.  111-23; 

“Weapon  Systems  Acquisition  Reform  Act  of 2009”),  Mr.  Jeff  Loren,  SAF/AQR  (Alion  Science  &  Technology) 

•  8846  -  Air  Force  Materiel  Command  Early  Systems  Engineering,  Dr.  Brian  Kowal,  USAF 

•  9016-^4  Framework  for  Enhancing  Forward-looking  Capability  Delivery  Metrics,  Mr.  Leonard  Sadauskas,  DoD  CIO  CT&S 

•  9082  -  Including  Environment,  Safety,  and  Occupational  Health  (ESOH)  Requirements  in  Joint  Capabilities  Integration  and  Development 
System  (JCIDS)  Documents,  Mr.  Sherman  Forbes,  U.S.  Air  Force 

•  8835  -  T&E  Collaboration  and  Contributions  during  Early  Program  Acquisition,  Mr.  Stephen  Scukanec,  Northrop  Grumman  Corporation 
Aerospace  Systems 

•  8795  -  Mission-based  Test  and  Evaluation  Strategy:  Creating  Linkages  between  Technology  Development  and  Mission  Capability,  Mr.  John 
Beilfuss,  U.S.  Army  Research  Laboratory 

•  Panel  Topic:  8924,  8925 , 8933  -  Early  Systems  Engineering  in  DoDI  5000.02,  Dr.  Judith  Dahmann,  Ms.  Lisa  Reuss,  Ms.  Sharon  Vannucci, 
Systems  Engineering  Directorate,  ODDR&E 

•  8949  -  Updated  DoD  5000  and  CJCS  31 70  Policies:  A  Requirements  to  Acquisition  Gap  Analysis,  Mr.  John  Lohse,  Raytheon  Company 

•  88 1 3  -  Emerging  Roles  for  Systems  Engineering  in  Defense  Decision  Making;  Better  Aligning  Requirements  and  Acquisition  with  the  Budget 
and  Security  Environments,  Mr.  Vincent  Roske,  Institute  for  Defense  Analyses 

•  9026  -  Early  SE  Determination  of  Best-Fit  System  Life  Cycle  Processes,  Dr.  Barry  Boehm,  USC 

TRACK  3  -  Technology  Maturity  -  Bayview  I 

•  8991  -  Systems  Engineering  for  the  Science  &  Technology  Community,  Mr.  Russell  Menko,  U.  S.  Army  RDECOM/TARDEC 

•  9017  -  Linking  Systems  Engineering  Artifacts  with  Complex  Systems  Maturity  Assessments,  Dr.  Brian  Sauser,  Stevens  Institute  of  Technology 

•  8770  -  Incorporating  Maturity  Assessment  into  House  of  Quality  for  Improved  Decision  Support  Analysis  and  Risk  Management,  Mr.  Pavel 
Fomin,  U.S.  Air  Force 

•  8798  -  The  New  Technology  Readiness  Assessment  Process,  Dr.  Jay  Mandelbaum,  Institute  for  Defense  Analyses 

•  8870  -  S&T  Portfolio  Maturity  &  Performance  Analysis:  The  Concept  of  Critical  Research  Elements,  Mr.  Has  Patel,  Infologic,  Inc. 

•  8879  -  TRL  Vectors  in  IPPD-based  Portfolio  Management,  Mr.  Michael  Bartmess,  General  Dynamics/ AIS 

•  8963  -  Air  Force  Concept  Maturity  Assessment,  Mr.  George  Freeman,  U.S.  Air  Force,  Center  for  Systems  Engineering 

•  8900  -  DOD ’s  Weapon  System  Portfolio:  Are  Results  Getting  Any  Better?,  Mr.  Michael  Sullivan,  U.S.  Government 

•  8894  -  Air  Force  Initiative  -  High  Confidence  Technology  Transition  Planning  Through  the  Use  of  Stage-Gates  -  Update,  Mr.  Randy  Bullard, 
U.S.  Air  Force  Materiel  Command 

•  8891  -  A  comprehensive  overview  of  techniques  for  measuring  system  readiness,  Mr.  James  Bilbro,  JB  Consulting  International 

TRACK  4  -  Test  and  Evaluation  of  SOS  -  Mission  I 

•  8825  -  Test  and  Evaluation  in  a  System  of  Systems  Environment,  Mr.  Edwin  McDermott,  653  ELSW,  Electronic  Systems  Center 

•  8849  -  Joint  Integration  and  Interoperability  Lab  (JSIIL),  Mr.  Steven  Whitehead,  SL,  J8  Technical  Director,  USJFCOM 

•  8935  -  Systems  of  Systems  Systems  Engineering  and  Test  and  Evaluation,  Dr.  Judith  Dahmann,  Systems  Engineering  Directorate, 
ODDR&E/MITRE 

TRACK  4  -  Practical  Systems  Engineering  -  Mission  I 

•  9014  -  SAVI:  Aerospace  Platform  Development  and  Certification  Using  Modeling  and  Simulation  to  “Integrate,  then  Build”,  Mr.  Gregory 
Pollari,  Rockwell  Collins 

•  8855  -  Certify  and  Fly  Right:  Preparing  for  DO-297  Certification,  Mr.  Ketih  Custer,  Esterline  Control  Systems- A  VIST  A 

•  8973  -  C-17  Transition  to  Criteria-based  Airworthiness  Certification,  Mr.  Christian  Stillings,  USAF  516  AESG 

TRACK  4  -  Test  and  Evaluation  -  Mission  I 

•  8848  -  Integrated  Testing:  We  Can  Do  It,  Dr.  Beth  Wilson,  Raytheon  Company 

•  8882  -  Test  &  Evaluation  Strategy  for  the  Technology  Development  Phase,  Ms.  Darlene  Mosser-Kemer,  OUSD(AT&L)/DDR&E/DT&E 

•  8883  -  Test  &  Evaluation  Products  for  the  Systems  Engineering  Reviews,  Mr.  Woody  Eischens,  OUSD(AT&L)/DDR&E/DT&E 

•  88 14  -  Joint  Mission  Environment  Test  Capability  (JMETC),  Lowering  technical  Risk  by  Improving  Distributed  Test  Capabilities,  Mr.  Chip 
Ferguson,  JMETC 

•  8901  -  Review  Results  of  the  NDIA/OSD  Software  Test  Summit/Workshop,  Mr.  Thomas  Wissink,  Lockheed  Martin  IS&GS 

TRACK  5  -  Human  Systems  Integration  -  Mission  II 

•  8998  -  Human  Systems  Integration  -  Ensuring  the  Human  is  Considered  “Left  of  A  ”,  Col  Larry  Kimm,  USAF,  U.S  Air  Force 

•  8885  -  Human  Systems  Integration  (HSI)  -  Integrating  Human  Concerns  into  Life  Cycle  Systems  Engineering,  Ms.  Cynthia  Shewed,  Booz  Allen 
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Hamilton 

•  9012  -  Human  Systems  Integration:  Defining  and  Validating  a  Framework  for  Enhanced  Systems  Development,  Dr.  Matthew  Risser,  Pacific 
Science  &  Engineering  Group 

•  8975  -  What  is  Human  Systems  Integration  (HSI)  and  why  should  we  do  it?,  Mr.  Stuart  Booth,  Systems  Engineering  Directorate,  ODDR&E 

•  9042  -  Bounding  the  Human  Within  the  System,  Mr.  Michael  Mueller,  U.S.  Air  Force  Center  for  Systems  Engineering 

•  8829  -  The  Army  Health  Hazard  Assessment  Program ’s  Medical  Cost  Avoidance  Model,  Dr.  Timothy  A.  Kluchinsky,  Department  of  Army 

TRACK  5  -  System  Safety  -  ESOH  -  Mission  II 

•  9095  -  Using  Proposed  MIL-STD-882  Change  1  For  Hazardous  Materials  Management,  Ms.  Karen  Gill,  Booz  Allen  Hamilton 

•  8890  -  Building  Safer  UGVs  with  Run-time  Safety  Invariants,  Mr.  Michael  Wagner,  Carnegie  Mellon  University,  NREC 

•  Sherman  Forbes,  U.S.  Air  Force 

•  882D  -  Overview  of  Draft  MIL-STD-882D  With  Change  1,  Mr.  Bob  Smith,  Booz  Allen  Hamilton 

•  9070  -  Improving  Safety  Technology  Insertion  in  DoD  Acquisition  Programs ,  Dr.  Elizabeth  Rodriguez- Johnson,  Systems  Engineering 
Directorate,  ODDR&E 

•  9094  -  DoD  Green  Procurement  Program  Update  and  Path  Forward ,  Mr.  David  Asiello,  Office  of  the  Secretary  of  Defense 

•  909 1  -  Environment,  Safety,  and  Occupational  Health  (ESOH)  Risk  and  Technology  Requirements  Reporting  at  Acquisition  Program  Reviews , 
Ms.  Lucy  Rodriguez,  Booz  Allen  Hamilton 

TRACK  6  -  System  of  Systems  -  Mission  III 

•  8898  -  Designing  Collaborative  Systems  of  Systems  in  support  of  Multi-sided  Markets ,  Mr.  Philip  Boxer,  SEI 

•  8892  -  SysML  Strategies  to  Characterize  and  Analyze  Systems  of  Systems ,  Dr.  Jo  Ann  Lane,  University  of  Southern  California 

•  9041  -  On  Modeling  and  Simulation  Methods  for  Capturing  Emergent  Behaviors  for  Systems  of  Systems ,  Dr.  Jack  Zentner,  Georgia  Tech 
Research  Institute 

•  9060  -  M&S  Support  for  SoS  SE,  Dr.  Joann  Lane,  USC 

•  8776  -  The  Modular  SOS  Paradigm:  an  Availability  Paradox ?,  Mr.  Peter  Gentile,  Northrop  Grumman  Corporation 

•  8866  -  Extending  FMECA  to  Systems  of  Systems,  Mr.  Leopoldo  Mayoral,  Johns  Hopkins  University/ APL 

•  NDIA  SoS  Committee  Report ,  Dr.  Judith  Dahman,  Systems  Engineering  Directorate,  ODDR&E/MITRE 

•  8960  -  A  Distillation  of  Lessons  Learned  from  Complex  System  of  Systems  Acquisitions,  Dr.  Richard  Turner,  Stevens  Institute 

•  8784  -  Establishing  a  Departmental-Level  Systems-of-Sy stems  Engineering  Management  Construct  for  the  Department  of  the  Navy,  Progress 
Report ,  Mr.  John  Kevin  Smith,  Asst  Sec  of  the  Navy  for  RD&A,  Chief  Engineer 

•  8942  -  DoD  Systems  of  Systems  Update,  Dr.  Judith  Dahmann,  Systems  Engineering  Directorate,  ODDR&E/MITRE 

•  8961  -  Engineering  Systems  of  Systems:  An  Integration  Perspective,  Dr.  Emmett  Maddry,  NSWCDD 
TRACK  7  -  Program  Management  -  Palm  I 

•  8979  -  Boots  on  the  Ground:  Tactical  Planning  at  Program  Start  Up,  Mr.  Gerry  Becker,  Harris  Corporation 

•  8999  -  Program  Signature  Measurement,  Mr.  James  Thompson,  Systems  Engineering  Directorate,  ODDR&E 

•  9103  -  The  Economics  of  CMMI,  Mr.  Geoff  Draper,  Harris  Corporation 

•  8995  -  Integrated  Systems  Engineering  and  Developmental  Test  and  Evaluation,  Mr.  Chris  DiPetto,  DUSD(A&T)/SSE 

•  902 1  -  Critical  Success  Factors  for  Milestone  Review  Risk  Identification,  Dr.  Barry  Boehm,  USC 

•  9030  -  Lessons  Learned  in  Motivating  Software  Engineering  Process  Group  to  Focus  on  Achieving  Business  Goals  and  Not  on  Just  Achieving  a 
Maturity  Level,  Mr.  Girish  Seshagiri,  Advanced  Information  Services  Inc. 

•  9003  -  CMMI®  for  Executives,  Mr.  Geoff  Draper,  Harris  Corporation 

•  9034  -  Sustainment  and  Continued  Institutionalization  of  Best  Practices  and  CMMI®  at  SPAWAR,  Mr.  Michael  Kutch,  SPAWAR  Systems 
Center  Atlantic 

•  8791  -  Cost  and  Risk  Impacts  of  the  New  DOD  5000  Defense  Acquisition  Framework,  Dr.  Peter  Hantos,  The  Aerospace  Corporation 

•  8895  -  A  Comprehensive  Review  of  Maturity  Assessment  Approaches  for  Improved  Defense  Acquisition,  Ms.  Nazanin  Azizian,  The  George 
Washington  University 

TRACK  8  -  Net-Centric  Operations/Interoperability  -  Palm  II 

•  8874  -  The  Boeing  System  of  Systems  Engineering  (SoSE)  Process  and  Its  Use  in  Developing  Legacy-Based  Net-Centric  Systems  of  Systems,  Mr. 
John  Palmer,  The  Boeing  Company 

•  9010  -  Network  Enabled  Weapons,  A  System  Engineering  Approach  to  Achieve  Inter operabilty,  Mr.  Andrew  Lieux,  Naval  Air  Warfare  Center 
Weapons  Division 

•  8854  -  Human  Interoperability  Enterprise  and  Net  Centric  Operations,  Mr.  Jack  Zavin,  ASD  (Nil) 

•  8780  -  Net-Centric  Best  Practices ,  Mr.  Hiekeun  Ko,  JPEO-CBD  -  Software  Support  Activity 

•  8788  -  Data  sharing  in  a  Stability  Operations  Community  of  Interest:  Utilizing  a  pilot  program  to  prove  concepts  and  develop  trust.,  Mr.  Gerald 
Christman,  Femme  Comp  Inc. 

•  8853  -  C4I  Architecture  for  Joint  ASW,  Mr.  Gregory  Miller,  Naval  Postgraduate  School 

•  8929  -  Extending  Net-Centric  Quality  of  Service  to  Systems  of  Systems ,  Maj  Vinod  Naga,  USAF,  Air  Force  Institute  of  Technology 

•  9081  -  Testing  in  Service- oriented  Environments ,  Mr.  Soumya  Simanta,  SEI 

•  8913  -  Linking  Interoperability  and  Measures  of  Effectiveness:  A  Method  for  Evaluating  Architectures,  Dr.  David  Jacques,  Air  Force  Institute 
of  Technology 

TRACK  8  -  Speciality  Engineering  -  Palm  II 

•  8944  -  DoD ’s  Refocus  on  Specialty  Engineering  (Reliability,  Availability  and  Maintainability;  Producibility  and  Quality,  Supportability,  Safety 
and  Human  Systems  Integration),  Mr.  Chester  Bracuto,  Systems  Engineering  Directorate,  ODDR&E 

•  9043  -  Implementing  the  Materiel  Availability  KPP  in  DoD  acquisition  programs — balancing  life-cycle  costs  with  warfighter  needs,  Mr.  Grant 
Schmieder,  Systems  Engineering  Directorate,  ODDR&E 

•  8873  -  IUID  enables  streamlined  acquisition  and  system  engineering,  Mr.  Robert  Leibrandt,  DoD  UID  Policy  Office 

•  8958  -  Security  Systems  Engineering,  Mrs.  Kristen  Baldwin,  Systems  Engineering  Directorate,  ODDR&E 
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THURSDAY.  OCTOBER  29.  CONCURRENT  SESSIONS 

TRACK  1  -  Systems  Engineering  Effectiveness  -  Bayview  III 

•  8887  -  Achieving  a  Systems  Engineering  Culture  in  a  Science  and  Technology  Laboratory  Environment,  Mr.  Robert  Rapson,  Materials  and 
Manufacturing  Directorate,  AFRL 

•  8920  -  A  Methodology  for  Assessing  Systems  Engineering  Practices,  Ms.  Lauren  Levy,  Johns  Hopkins  University/ APL 

•  9097  -  Acquisition  ESOH  Risk  Management-How  to  Make  It  Work,  Mr.  Bob  Smith,  Booz  Allen  Hamilton 

TRACK  1  -  Architecture  -  Bayview  III 

•  883 1  -  Human-Centered  Design  in  Systems  Engineering:  Human  View  Methodology,  Dr.  Robert  Smillie,  SPAWAR 

•  8830  -  Systems  Engineering  Needs  of  the  DoDAF  -  Report  of  the  Architecture  Frameworks  Working  Group,  Mr.  Joe  Kuncel,  Northrop 

Grumman  Corporation 

•  8824  -  Delivering  DoDAF  Version  2. 0  to  Architects  and  Systems  Engineers  for  IT  Systems  and  Services ,  Mr.  Walt  Okon,  Department  of 
Defense,  CIO,  Enterprise  Architecture 

•  897 1  -  Advancing  Systems  Engineering  Practice  using  Model  Based  System  Development,  Mr.  Sanford  Friedenthal,  Lockheed  Martin 
Corporation 

•  9004  -  Evolving  Systems  Engineering  through  Model  Driven  Functional  Analysis,  Dr.  Mark  Blackburn,  Systems  and 

TRACK  2  -  Logistics  Systems  -  Bayview  II 

•  9063  -  An  Integrated  RAM  Approach  to  System  Design  and  Support,  Mr.  Robert  Finlayson,  Johns  Hopkins  University/ APL 

•  903 1  -  Supportability  Lessons  Learned  with  Line  Replaceable  Modules,  Ms.  Heity  Hsiung,  Raytheon  Company 

•  8908  -  Successful  First  AESA  Deployment  through  Application  of  System  Engineering,  Mr.  Scott  Nichols,  Raytheon  Company 

•  9039  -  Applying  Systems  Engineering  to  Fielded  Weapon  Systems  and  End-Items,  Mr.  Michael  Ucchino,  AF  Center  for  Systems  Engineering 

•  9008  -  Upgrade  Fluid  System  Filter  Element  Monitoring  To  Increase  Operational  Reliability  and  Support  Condition  Based  Maintenance 
Capability,  Mr.  Gary  Rosenberg,  Constellation  Technology  Corportation 

•  8834  -  Tailoring  Systems  Engineering  for  Technical  Support  of  Legacy  Products,  Mr.  Joseph  Skandera,  BAE  Systems 

•  9092  -  The  role  of  simulation  in  tracking  mobile  assets  using  RFID  technology,  Mr.  Swee  Leong,  National  Institute  of  Standards  and 
Technology 

TRACK  3  -  Modeling  &  Simulation  -  Bayview  I 

•  8939  -  Understanding  the  New  DoD  Instruction  5000.61:  ‘DoD  Modeling  &  Simulation  Verification,  Validation  and  Accreditation  (VV&A)  ”, 
Mr.  Michael  Truelove,  Systems  Engineering  Directorate,  ODDR&E 

•  8950  -  Live,  Virtual,  Constructive  Architecture  Roadmap:  The  Quest  for  Interoperability,  Standards,  and  Reuse,  Dr.  Gary  Allen,  Joint  Training 
Integration  &  Evaluation  Center 

•  9048  -  Revisions  to  the  Acquisition  Modeling  &  Simulation  Master  Plan,  Mr.  Stephen  Swenson,  Systems  Engineering  Directorate,  ODDR&E 

•  8759  -  A  Systems  Engineering  Framework  for  Integrating  M&S  Development  Best  Practices,  Dr.  Katherine  Morse,  Johns  Hopkins 
University/APL 

•  9052  -  Best  Practices  in  Contracting  for  Models,  Simulations,  and  Associated  Data,  Mr.  Dennis  Shea,  CNA 

•  8947  -  Report  on  a  Study  on  Management  Concepts  for  Broadly-Needed  Modeling  and  Simulation  Tools  in  the  US.  Department  of  Defense,  Dr. 
James  Coolahan,  Johns  Hopkins  University/APL 

•  8836  -  Producibility  Modeling  &  Simulation  Needs  for  Early  Systems  Engineering  Evaluations  of  Alternative  Design  Concepts,  Dr.  A1  Sanders, 
Honeywell  Aerospace 

•  8810  -  Using  Simulation  to  Define  and  allocate  probabilistic  Requirements,  Ms.  Yvonne  Bijan,  Lockheed  Martin  Aeronautics 

•  8923  -  Integration  of  Operational  Simulations  With  Physics-Based  Models  For  Engineering  Analysis,  Mr.  Stephen  Guest,  Lockheed  Martin 
Aeronautics 

TRACK  4  -  Practical  Systems  Engineering  -  Mission  I 

•  8980  -  Using  Model-driven  Engineering  Techniques  for  Integrated  Flight  Simulation  Development,  Mr.  Douglas  Fiehler,  Raytheon  Missile 
Systems 

•  9007  -  Technology  Maturation  for  the  Automated  Aerial  Refueling  (AAR)  Project,  Ms.  Carol  Ventresca,  SynGenics  Corporation 

•  8880  -  Naval  Postgraduate  School  Advanced  Seabase  Enabler  Project:  A  Systems  Engineering  Case  Study,  Mr.  Lance  Flitter,  NSWC, 

Carderock  Division 

•  8946  -  Protecting  the  Mission,  Preserving  Legacy  and  Promoting  Growth,  Ms.  Patti  Scaramuzzo,  Lockheed  Martin  Corporation 

•  9054  -  A-10  Avionics  System  Architecture  Trade  Study  and  Analysis  (A  VSATA)  Program,  Mr.  Richard  Sorensen,  KIHO  Military  Acquisition 

Consulting,  Inc. 

•  8976  -  A  Systems  Engineering  Model  for  Roadmap  Alignment,  Mr.  Si  Dok,  U.  S.  Army  TARDEC 

•  9080  -  Rapid  Systems  Engineering  of  the  MRAP  Gunner  Restraint  System  Saves  Lives,  Ms.  Michelle  Bowen,  JPO  MRAP 

•  9002  -  Key  Considerations  for  Building  Highly  Available,  Mission-Critical  Systems,  Mr.  Stephen  Mills,  GoAhead  Software 

TRACK  5  -  Human  Systems  Integration  -  Mission  II 

•  8937  -  Integrating  the  Human  into  the  system,  integrating  HSI  Tools  into  Systems  Engineering,  Dr.  Jennifer  Narkevicius,  Jenius  LLC 

•  9064  -  Economics  of  Human  Systems  Integration:  Early  Life  Cycle  Cost  Estimation  Using  HSI  Requirements,  2ndLt  Kevin  Liu,  USMC,  MIT 

•  Proccess  Management  and  tool  selection  to  minimize  risk  of  hand-arm  vibration  syndrome ,  Mr.  Sherman  Forbes,  U.S.  Air  Force 

TRACK  5  -  Systems  Engineering  Development  Environment  -  Mission  II 

•  8945  -  Standards  Based  Development  Environment,  Mr.  Christopher  Oster,  Lockheed  Martin  Corporation 

•  8922  -  The  Role  of  DoD  in  Systems  Engineering  Standards  and  Models,  Mr.  Donald  Gantzer,  Systems  Engineering  Directorate,  ODDR&E 

•  8844  -  The  Power  of  the  Spec:  Understanding  the  Many  Diverse  Roles  in  SE  of  Good  Specifications  &  Standards.  ”,  Mr.  Robert  Kuhnen,  U.S. 
Air  Force 
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•  8967  -  Generating  Visual  and  Interactive  Output  from  System  Engineering  Tools,  Mr.  John  Schatz,  Systems  and  Proposal  Engineering  Company 

•  9015  -  Challenges  and  Benefits  of  applying  ISO  STEP,  Mr.  Stuart  Booth,  Systems  Engineering  Directorate,  ODDR&E 

•  9059  -  Smalls  at  Conceptual  Design  Trade  and  Cost  Modeling  Tool,  Dr.  Deganit  Armon,  Advatech  Pacific,  Inc 

TRACK  6  -  Enterprise  Health  Management  -  Mission  III 

•  8815  -  Applying  Systems  Engineering  to  Operational  System  Improvements,  Ms.  Ryanne  Gentry,  Acquisition  Logistics  Engineering 

•  8842  -  Applications  in  Integrated  Diagnostics,  Mr.  Jimmy  Simmons,  Georgia  Tech  Research  Institute 

•  8884  -  Tactical  Wheeled  Vehicle  Integrated  Diagnostics ,  Mr  Lawrence  Osentoski,  DRIVE  Developments,  Inc. 

TRACK  6  -  System  of  Systems  -  Mission  III 

•  8964  -  Software  Assurance  in  a  System  of  Systems  World:  Interoperability  Challenges  -  Reports  from  the  Field,  Dr.  Carol  Sledge,  SEI 

•  8969  -  An  Introduction  to  Influence  Maps:  Foundations,  Construction,  and  Use,  Mr.  James  Smith,  SEI 

•  9024  -  Dynamic  Modeling  of  Programmatic  and  Systematic,  Dr.  Brian  Sauser,  Purdue  University 

•  8915  -  System  of  Systems  Challenges  and  Solutions:  Case  Study  Insights,  Mr.  John  Colombi,  U.S.  Air  Force  Institute  of  Technology 

TRACK  7  -  Work  Force  Development  -  Palm  I 

•  8966  -  Improving  Systems  Engineering  Curriculum  Using  a  Competency-Based  Assessment  Approach,  Ms.  Alice  Squires,  Stevens  Institute  of 
Technology 

•  9088  -  Enhancing  Systems  Engineering  Competencies  in  the  Enterprise,  Mr.  Gary  Roedler,  Lockheed  Martin  Corporation 

•  8789  -  Achieving  Acquisition  Excellence  via  Improving  the  Systems -Engineering  Workforce,  Dr.  Kenneth  Nidiffer,  SEI 

•  8926  -  Systems  Engineering  Workforce  Development  Update,  Dr.  Don  Gelosh,  Systems  Engineering  Directorate,  ODDR&E 

•  9076  -  Assessing  Systems  Engineering  Personnel  Competency:  Framework  and  Tool  Experience,  Dr.  Barry  Boehm,  University  of  Southern 
California 

•  8943  -  Team  SE  Skill  Set,  Mr.  Charles  Garland,  U.S.  Air  Force  Center  for  Systems  Engineering 

•  8956  -  Systems  Engineering  Approach  to  Workforce  Development,  Mr.  James  Miller,  U.S.  Air  Force 

•  9046  -  Developing  an  Introductory  Systems  Engineering  Practitioners  Course:  “Model-Based  Systems  Engineering  (MBSE)  With  SysML  ”,  Mr. 
Joseph  Wolfrom,  Johns  Hopkins  University/ APL 

•  8878  -  Advanced  Simulation  Course  for  Army  Simulation  Management  Professionals,  Dr.  Gene  Paulo,  Naval  Postgraduate  School 

TRACK  8  -  Software  Intensive  Systems  -  Palm  II 

•  8977  -  Overview  of  DoD  Software  Engineering  Initiatives,  Mr.  Scott  Lucero,  Systems  Engineering  Directorate,  ODDR&E 

•  8820  -  Graduate  Software  Engineering  Reference  Curriculum  (GSwERC),  Ms.  Nicole  Hutchison,  Analytic  Services,  Inc. 

•  8739  -  Quality  Assessment  of  Software-Intensive  System  Architectures  and  their  Requirements  (QUASAR),  Mr.  Donald  Firesmith,  SEI 

•  8812  -  ^4  Systems  Engineering  Approach  to  Multi-Level  Security  in  a  Service  Oriented  Architecture,  Mr.  Timothy  Greer,  Lockheed  Martin 
Corporation 

•  9104  -  Static  Code  Analysis:  Best  Practices  for  Software  Assurance  in  the  Acquisition  Life  Cycle,  Mr.  Paul  Croll,  CSC 

•  8996  -  Engineering  Improvement  in  Software  Assurance:  A  Landscape  Framework,  Ms.  Lisa  Brownsword,  SEI 

•  8802  -  Open  Source  Technology  for  Enterprise  Health  Management,  Mr.  Edward  Beck,  CSC 

•  8901  -  Review  Results  of  the  NDIA/OSD  Software  Test  Summit/Workshop,  Mr.  Thomas  Wissink,  Lockheed  Martin  IS&GS 

•  9506  -  Software  Acquisition  Management  Practical  Experience,  Mr.  James  Jones,  SSAI 
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SYSTEMS  ENGINEERING  CONFERENCE 
CONFERENCE  OVERVIEW 


►  CONFERENCE  OVERVIEW 

The  NDIA  Systems  Engineering  conference  is  focused  on  improving  acquisition  and  performance  of  Defense 
programs  and  systems,  including  net-centric  operations  and  data/information  interoperability,  system-of- 
systems  engineering  and  all  aspects  of  system  sustainment.  Convened  in  San  Diego,  CA,  October  26-29,  2009, 
this  conference  is  sponsored  by  the  National  Defense  Industrial  Association,  Systems  Engineering  Division, 
with  technical  co-sponsorship  by  IEEE  AES,  IEEE  Systems  Council  and  the  International  Council  on  Systems 
Engineering,  and  is  supported  by  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics,  Office  of  the  Director,  Defense  Research  &  Engineering/ Systems  Engineering. 

►  BACKGROUND 

The  Department  of  Defense  continues  to  work  to  improve  the  acquisition  of  military  equipment  and  capability 
to  assist  the  warfighter  in  protecting  the  U.S.  and  its  allies,  and  help  oppressed  nations  around  the  world,  amidst 
continuously  changing  conditions  and  threats.  The  DoD  seeks  to  improve  the  acquisition  process  and  overall 
program  execution  of  military  systems,  to  provide  greater,  more  effective  and  reliable  warfighting  capability,  at 
affordable  cost  and  within  reasonable  schedules.  One  of  the  primary  and  critically  important  areas  of  program 
acquisition  and  execution  lies  in  the  umbrella  discipline  of  Systems  Engineering,  which  is  the  overall  integrating 
function  in  defense  programs,  from  proper  requirements  definition  &  flowdown,  effective  and  affordable  design 
that  integrates  reliability,  availability  and  maintainability  considerations  into  the  overall  balance  of  design 
that  emphasizes  supportability  and  usage  aspects  along  with  overall  performance,  cost  and  schedule.  Systems 
Engineering  principles  embody  strong  technical  and  risk  management  aspects,  for  both  the  acquiring  program 
office  as  well  as  the  executing  defense  prime  and  subcontractors.  Strong  emphasis  on  Systems  Engineering 
throughout  the  life  cycle  of  the  program,  from  concept  development  through  sustainment,  is  a  key  enabler  of 
successful  programs.  The  annual  Systems  Engineering  Conference  explores  the  role  of  Systems  Engineering  in 
defense  programs  from  all  aspects  and  perspectives,  including  the  pragmatic,  practical  and  academic  viewpoints, 
and  brings  key  practitioners  together  to  work  on  effective  solutions  to  achieving  a  successful  warfighting  force. 

►  CONFERENCE  OBJECTIVES 

This  conference  seeks  to  create  an  interactive  forum  for  Program  Managers,  Systems  Engineers,  Chief  Scientists 
and  Engineers  and  Managers  from  the  Requirements,  Design,  Verification,  Support,  Logistics  and  Test 
communities  from  Government,  Academia,  and  Industry.  The  conference  will  provide  the  opportunity  to 
shape  policy  and  procedures  by  exchanging  innovative  tactics  and  lessons  learned. 


SYSTEMS  ENGINEERING  CONFERENCE 
ATTENDEE  INFORMATION 


►  CONTACTS 

Technical  Program 
Co-Chairs: 

Mr.  Steve  Henry, 

Manager,  Systems  Engineering 
and  Program  Support, 
Northrop  Grumman 
Information  Systems, 
stephen.henry@ngc.com, 

(703)  361-3724 


►  ATTIRE 

Appropriate  dress  for  this  conference  is  business  casual  for  civilians  and  class 
B  uniform  for  military.  During  conference  registration  and  check-in,  each 
participant  will  be  issued  an  identification  badge.  Please  be  prepared  to 
present  a  picture  ID.  Badges  must  be  worn  at  all  conference  functions. 

►  CONFERENCE  PROCEEDINGS 

Proceedings  will  be  available  on  the  web  through  the  Defense  Technical 
Information  Center  (DTIC),  and  will  be  available  one  to  two  weeks  after 
the  conference.  You  will  receive  notification  via  e-mail  once  proceedings  are 
posted  and  available  on  the  web. 


Dr.  Tom  Christian, 

ASC/EN, 

thomas.christian@wpafb.af.mil, 

(478)  926-2457 

Conference  Chair: 

Mr.  Bob  Rassa, 

Director,  Systems 
Supportability,  Raytheon; 
Chair,  Systems  Engineering 
Division,  NDIA, 
rcrassa@raytheon.com, 

(310)  985-4962 

Meeting  Planner: 

Ms.  Suzanne  Havelis, 

NDIA,  shavelis@ndia.org, 
(703)  247-2570. 

Conference  Director: 

Mr.  Sam  Campagna, 

NDIA,  scampagna@ndia.org, 
(703)  247-2544 


►  CONTINUING  EDUCATION  UNIT  CREDIT 

NDIA  is  offering  CEU  credit  options  for  the  Systems  Engineering 
Conference.  For  more  information,  please  contact  Ms.  Suzanne  Havelis  at 
703.247.2570  or  shavelis@ndia.org. 

►  2010  CALL  FOR  PAPERS  INFORMATION 

The  primary  objective  of  the  13th  Annual  Systems  Engineering  Conference 
is  to  provide  insight,  information  and  lessons  learned  into  how  we  can 
improve  the  overall  performance  of  defense  programs  via  a  better,  more 
focused  application  of  systems  engineering  that  will  lead  to  more  capable, 
interoperable  and  supportable  weapon  systems  for  the  warfighter,  with 
reduced  total  ownership  costs,  to  help  our  military  meet  its  current  and  new 
mission  area  and  capabilities  requirements.  Technical  and  management 
presentations  are  a  key  tactic  in  achieving  this  objective.  You  are  invited  to 
submit  a  short  (under  300  word)  abstract  of  a  presentation  for  a  session  (see 
topics  on  the  website).  Abstracts  must  fully  describe  the  planned  content 
and  how  the  presentations  will  advance  the  objectives  of  the  conference  and 
session.  All  accepted  presentations  will  be  delivered  at  the  conference  in 
electronic  format;  full  papers  are  optional  and  are  not  required. 

Abstracts  must  include  the  following  administrative  information: 
presentation  title,  authors  name,  title,  e-mail  address,  phone  number, 
mailing  address  and  organization  and  the  conference  session  targeted. 
Abstracts  must  be  submitted  no  later  than  Sunday,  May  30,  2010  via  the 
following  web  link: 

http://application.ndia.org/abstracts/1870 

Abstracts  will  only  be  accepted  through  this  web  link,  and  all  required 

information  must  be  completed.  Upon  completion 

of  the  required  information,  you  will  receive  an  e-mail  confirmation. 

**Conference  presenters  are  not  exempt  from  registration  and  conference 
fees. 
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OVERVIEW:  OCTOBER  25-0CT0BER  27 

CONFERENCE  AGENDA 

SUNDAY,  OCTOBER  25,  2009 

5:00  pm  -  7:00  pm  REGISTRATION  FOR  TUTORIALS  AND  GENERAL  CONFERENCE 


MONDAY,  OCTOBER  26,  2009 


7:00  am  -  6:00  pm 
7:00  am  -  8:00  am 
8:00  am  -  12:00  pm 
9:45  am  -  10:15  am 
12:00  pm  -  1:00  pm 
1:00  pm -5:00  pm 
2:45  pm  -  3:15  pm 
5:00  pm  -  6:00  pm 


REGISTRATION 

CONTINENTAL  BREAKFAST  (FOR  TUTORIAL  ATTENDEES  ONLY) 

TUTORIAL  TRACKS 

MORNING  BREAK  (FOR  TUTORIAL  ATTENDEES  ONLY) 

LUNCH  (FOR  TUTORIAL  ATTENDEES  ONLY) 

TUTORIAL  TRACKS  CONTINUED 

AFTERNOON  BREAK  (FOR  TUTORIAL  ATTENDEES  ONLY) 

RECEPTION  IN  THE  REGATTA  PAVILION  -  OPEN  TO  ALL  CONFERENCE  ATTENDEES 


TUESDAY,  OCTOBER  27,  2009 


7:15  am  -  7:00  pm 
7:15  am  -  8:15  am 
8:15  am  -  8:30  am 


REGISTRATION 

CONTINENTAL  BREAKFAST  IN  THE  REGATTA  PAVILION 
PLENARY  SESSION  1  -  INTRODUCTION  &  OPENING  REMARKS 


►  Mr.  Sam  Campagna,  Director ;  Operations ,  NDIA 

►  Mr.  Bob  Rassa,  Director ;  Systems  Supportability ;  Raytheon ;  Chair, 
Systems  Engineering  Division,  NDIA 


8:30  am  -  9:30  am  KEYNOTE 

►  Honorable  Zachary  J.  Lemnios,  Director,  Defense  Research  and  Engineering,  Office  of  the 
Under  Secretary  of  Defense  (Acquisition,  Technology  and  Logistics ) 


9:30  am  -  10:00  am  MORNING  BREAK  IN  THE  REGATTA  PAVILION 

10:00  am  -  12:00  pm  PLENARY  SESSION  2  -  ACQUISITION  EXECUTIVES  PANEL 

View  from  the  Top:  How  Can  SE  Support  Program  Execution ? 

Moderator:  Mr.  Terry  Jaggers,  Principal  Deputy,  Systems  Engineering,  Office  of  the  Director, 
Defense  Research  and  Engineering 

►  Mr.  David  G.  Ahern,  Director,  Por folio  Systems  Acquisition,  Office  of  the  Under  Secretary 
of  Defense  (Acquisition,  Technology  and  Logistics ) 

►  Mr.  Thomas  E.  Mullins,  Deputy  Assistant  Secretary  for  Plans,  Programs,  and  Resources 
(SAAL-ZR),  Office  of  the  Assistant  Secretary  of  the  Army  (Acquisition,  Logistics  and 
Technology) 

►  Mr.  Christopher  A.  Miller,  PEO  for  Command,  Control,  Communications,  Computers 
and  Intelligence  (C4I),  US.  Navy 

►  Mr.  Randall  G.  Walden,  Director,  Information  Dominance  Programs,  Office  of  the 
Assistant  Secretary  of  the  Air  Force  (Acquisition) 


12:00  pm  -  1:30  pm 


LUNCH  WITH  SPEAKER  IN  THE  REGATTA  PAVILION 

►  Mr.  Stephen  Welby,  Director,  Systems  Engineering,  Office  of  the  Director,  Defense 
Research  and  Engineering 


SYSTEMS  ENGINEERING  CONFERENCE 

OVERVIEW:  OCTOBER  27-0CT0BER  29 

TUESDAY,  OCTOBER  27,  2009  -  CONTINUED 


1:30  pm  -  3:15  pm 


3:15  pm -3:30  pm 
3:30  pm  -  5:15  pm 


PLENARY  SESSION  3  -  TEST  &  EVALUATION  EXECUTIVES  PANEL 

View  from  the  Top:  How  SE  Can  Support  Test  and  Evaluation ? 

Moderator:  Mr.  Jim  O’Bryon,  The  O’Bryon  Group;  Chair,  NDIA  Test  and  Evaluation  Division 

►  Dr.  James  N.  Streilein,  Technical  Advisor,  HQ  Army  Test  &  Evaluation  Command 

►  Ms.  Amy  Markowich,  Deputy  DoN  T&E  Executive 

►  Colonel  Dexter  M.  Sapinoso,  USAF,  Chief  of  Air  Force  Test  and  Evaluation  Policy  and 
Programs 

►  Mr.  Christopher  DiPetto,  Acting  Director,  DevelopmentalTest  and  Evaluation,  Office  of  the 
Director,  Defense  Research  and  Engineering 

AFTERNOON  BREAK  IN  THE  REGATTA  PAVILION 

PLENARY  SESSION  4  -  SE  AND  ACQUISITION  REFORM:  THE  WAY  AHEAD 

Moderator:  Mrs.  Kristen  Baldwin,  Systems  Engineering  Director  ate,  Off ce  of the  Director, 
Defense  Research  and  Engineering 

►  Mr.  Ross  Guckert,  Office  of  the  Secretary  of  the  Army  for  Acquisition,  Logistics  and 
Technology  (ASA(ALT)) 

►  Mr.  Carl  Siel,  Office  of  the  Assistant  Secretary  of  the  Navy  for  Research,  Development  and 
Acquisition  (ASNfRDA )  CHSENG) 

►  Colonel  Shawn  Shanley,  USAF,  Chief  Systems  Engineer,  Office  of  the  Assistant  Secretary  of  the 
Air  Force  for  Acquisition,  Science,  Technology,  and  Engineering  (SAF/AQR) 

►  Mr.  Nicholas  Torelli,  Systems  Engineering  Directorate,  Office  of  the  Director,  Defense  Research 
and  Engineering 


5:30  pm  -  7:00  pm  RECEPTION  IN  THE  REGATTA  PAVILION 

WEDNESDAY,  OCTOBER  28,  2009 


7:00  am  -  5:15  pm 
7:00  am  -  8:00  am 
8:00  am  -  12:00  pm 
9:45  am  -  10:15  am 
12:00  pm  -  1:30  pm 
1:30  pm  -  5:15  pm 
3:15  pm -3:30  pm 
5:15  pm 


REGISTRATION 

CONTINENTAL  BREAKFAST  IN  THE  REGATTA  PAVILION 

CONCURRENT  SESSIONS  -  Please  refer  to  the  following  pages  for  session  schedule 

MORNING  BREAK  IN  THE  REGATTA  PAVILION 
AWARDS  LUNCH  IN  THE  REGATTA  PAVILION 

CONCURRENT  SESSIONS  -  Please  refer  to  the  following  pages  for  session  schedule 

AFTERNOON  BREAK  IN  THE  REGATTA  PAVILION 
WEDNESDAY  SESSION  ADJOURNS 


THURSDAY,  OCTOBER  29,  2009 


7:00  am  -  3:00  pm 
7:00  am  -  8:00  am 
8:00  am  -  12:00  pm 
9:45  am  -  10:15  am 
12:00  pm  -  1:00  pm 
1:00  pm  -  3:00  pm 


REGISTRATION 

CONTINENTAL  BREAKFAST  IN  THE  REGATTA  PAVILION 

CONCURRENT  SESSIONS  -  Please  refer  to  the  following  pages  for  session  schedule 

MORNING  BREAK  IN  THE  REGATTA  PAVILION 
LUNCH  IN  THE  REGATTA  PAVILION 

CONCURRENT  SESSIONS  -  Please  refer  to  the  following  pages  for  session  schedule 


3:00  pm 


CONFERENCE  ADJOURNS 


MONDAY,  OCTOBER  26,  TUTORIAL  SESSIONS 


TRACK 

8:00  AM 
SESSION  A 

10:15  AM 
SESSION  B 

1:00  PM 
SESSION  C 

3:15  PM 
SESSION  D 

TRACK  8 

Palm  II 

8819  -  1A8  -  Tutorial: 

Rethinking  Risk  Management 

Ms.  Audrey  Dorofee,  SEI/ 

CMU 

8819  -  1B8  -  Tutorial: 

Rethinking  Risk  Management 

Ms.  Audrey  Dorofee,  SEI/CMU 

8877  -  1C8  -  Tutorial:  Best 
Practices  in  Modeling  and 
Simulation 

Dr.  Gene  Paulo,  Naval 
Postgraduate  School 

8877-  1D8 -Tutorial:  Best 
Practices  in  Modeling  and 
Simulation 

Dr.  Gene  Paulo,  Naval 
Postgraduate  School 

TRACK 7 

Palm  I 

8785  -  1A7  -  Tutorial:  Agile 
Development  in  Defense 
Acquisition 

8785  -  1B7  -  Tutorial:  Agile 
Development  in  Defense 
Acquisition 

8801  -  1C7- Tutorial: 

Integrating  SE  with  Earned 

Value  Management 

8801  -  1C7- Tutorial: 

Integrating  SE  with  Earned 

Value  Management 

Dr.  Peter  Hantos,  The 

Aerospace  Corporation 

Dr.  Peter  Hantos,  The  Aerospace 
Corporation 

Mr.  Paul  Soloman,  Performance- 
Based  Earned  Value 

Mr.  Paul  Soloman,  Performance- 
Based  Earned  Value 

v©  S 

y  .2 

9078  -  1A6  -  Tutorial: 
Organizational  Implications 
of  SoS 

9078  -  1B6  -  Tutorial: 
Organizational  Implications 
of  SoS 

8782  -  1C6  -  Tutorial: 

Technology  Transition  and  the 
Defense  Acquisition  System 

8782  -  1C6  -  Tutorial: 

Technology  Transition  and  the 
Defense  Acquisition  System 

H  S 

Ms.  Suzanne  Garcia,  SEI/CMU 

Ms.  Suzanne  Garcia,  SEI/CMU 

Mr.  William  Decker,  DAU 

Mr.  William  Decker,  DAU 

TRACK  5 

Mission  II 

8984  -  1A5  -  Tutorial:  How  to 
use  Lean  SE  Processes  to  Save 
Time  and  Money 

Mr.  Tim  Olson,  Lean  Solutions 
Institute,  Inc. 

8984-  1B5  -  Tutorial:  How  to 
use  Lean  SE  Processes  to  Save 
Time  and  Money 

Mr.  Tim  Olson,  Lean  Solutions 
Institute,  Inc. 

9072  -  1C5  -  Tutorial: 

Leveraging  the  Defense  Acq 
Program  Support  (DAPS) 
Methodology  to  Conduct 
Program  Assessment 

Mr.  Peter  Nolte,  Systems 
Engineering  Directorate, 
ODDR&E 

9072  -  1D5  -  Tutorial: 

Leveraging  the  Defense  Acq 
Program  Support  (DAPS) 
Methodology  to  Conduct 

Program  Assessment 

Mr.  Peter  Nolte,  Systems 
Engineering  Directorate, 
ODDR&E 

s«tl  i— i 

bl 

9035  -  1A4  -  Tutorial: 
Collaborative  Decision  Making 

9035  -  1B4  -  Tutorial: 
Collaborative  Decision  Making 

8931  -  1C4  -  Tutorial:  Role  of 
Mentoring  in  Developing  the 

Sys  Eng  Workforce 

8931  -  1D4  -  Tutorial:  Role  of 
Mentoring  in  Developing  the 

Sys  Eng  Workforce 

gi 

Dr.  Tommer  Ender,  Georgia 

Tech  Research  Institute 

Dr.  Tommer  Ender,  Georgia 

Tech  Research  Institute 

Mr.  Nicholas  Torelli,  Systems 
Engineering  Directorate, 
ODDR&E 

Mr.  Nicholas  Torelli,  Systems 
Engineering  Directorate, 
ODDR&E 

TRACK 3 

Bayview  I 

8955  -  1A3  -Tutorial:  Early 

Sys  Thinking  and  Planning  in 
WPN  Sys  Concept  Phase 

Mr.  Jeff  Loren,  SAF/AQR 
(Alion  Science  &  Technology) 

8955  -  1B3  -Tutorial:  Early  Sys 
Thinking  and  Planning  in  WPN 
Sys  Concept  Phase 

Mr.  Jeff  Loren,  SAF/AQR 
(Alion  Science  &  Technology) 

9040  -  1C3  -  Tutorial: 
Implementing  the  Materiel 
Availability  KPP  in  DoD 
Acquisition  Programs 

Mr.  Grant  Schmieder,  Systems 
Engineering  Directorate, 
ODDR&E 

9040  -  1D3  -  Tutorial: 
Implementing  the  Materiel 
Availability  KPP  in  DoD 
Acquisition  Programs 

Mr.  Grant  Schmieder,  Systems 
Engineering  Directorate, 
ODDR&E 

rs  a 

u  § 

8779  -  1A2  -  Tutorial:  Mission 
Based  Test  and  Eval  Strategy: 
Case  Study 

8779  -  1B2  -  Tutorial:  Mission 
Based  Test  and  Eval  Strategy: 

Case  Study 

8818-  1C2- Tutorial: 

Integrated  Testing  Enhances  SE 

8818  -  1D2  -  Tutorial: 

Integrated  Testing  Enhances  SE 

TRA 

Bayv 

Mr.  Christopher  Wilcox,  U.S. 
Army  Test  and  Evaluation 
Command 

Mr.  Christopher  Wilcox,  U.S. 
Army  Test  and  Evaluation 
Command 

Dr.  Beth  Wilson,  Raytheon 
Company 

Dr.  Beth  Wilson,  Raytheon 
Company 

TRACK  1 
Bayview  III 

8736  -1A1  -  Tutorial: 

Framework  of  Engineering 
Architectures 

8736-  1B1  -  Tutorial: 

Framework  of  Engineering 
Architectures 

8992  -1C1  -Tutorial:  SoS 

Quality  Attribute  Specification 
and  Architecture  Evaluation 

8992  -1D1  -Tutorial:  SoS 

Quality  Attribute  Specification 
and  Architecture  Evaluation 

Mr.  Donald  Firesmith,  SEI 

Mr.  Donald  Firesmith,  SEI 

Mr.  Michael  Gagliardi,  SEI 

Mr.  Michael  Gagliardi,  SEI 

WEDNESDAY,  OCTOBER  28,  CONCURRENT  SESSIONS 


11:25  AM 

8913  -  Linking 

Interoperability 

and  Measures  of 

Effectiveness:  A 

Method  for  Evaluating 

Architectures 

Dr.  David  Jacques, 

Air  Force  Institute  of 

Technology 

8895  -  A  Comprehensive 

Review  of  Maturity 

Assessment  Approaches 

for  Improved  Defense 

Acquisition 

Ms.  Nazanin  Azizian, 

The  George  Washington 

University 

8961  -  Engineering 

Systems  of  Systems:  An 

Integration  Perspective 

Dr.  Emmett  Maddry, 

NSWCDD 

9091  -  Environment, 

Safety,  and  Occupational 

Health  (ESOH) 

Risk  and  Technology 

Requirements  Reporting 

at  Acquisition  Program 

Reviews 

Ms.  Lucy  Rodriguez, 

Booz  Allen  Hamilton 

10:50  AM 

9081  -  Testing  in 
Service-oriented 
Environments 

Mr.  Soumya  Simanta, 

SEI 

8791  -  Cost  and  Risk 

Impacts  of  the  New 

DOD  5000  Defense 

Acquisition  Framework 

Dr.  Peter  Hantos,  The 

Aerospace  Corporation 

8942  -  DoD  Systems  of 

Systems  Update 

Dr.  Judith  Dahmann, 

Systems  Engineering 

Directorate,  ODDR&E/ 

MITRE 

9094  -  DoD  Green 

Procurement  Program 

Update  and  Path 

Forward 

Mr.  David  Asiello, 

Office  of  the  Secretary  of 

Defense 

10:15  AM 

8929  -  Extending 
Net-Centric  Quality  of 
Service  to  Systems  of 
Systems 

Maj  Vinod  Naga,  USAF, 
Air  Force  Institute  of 
Technology 

8982  -  Systemic  Root 
Cause  Analysis  -  Driving 
Improvements  into  the 
Acquisition  Process 

Mr.  Peter  Nolte,  Systems 

Engineering  Directorate, 

ODDR&EE) 

8840  -  Naval  Systems 

of  Systems  Engineering 

Guidebook  Update 

Ms.  Melinda  Reed,  DoD 

(ASN  RDA  CHSENG) 

9070  -  Improving  Safety 

Technology  Insertion 

in  DoD  Acquisition 

Programs 

Dr.  Elizabeth  Rodriguez- 

Johnson,  Systems 

Engineering  Directorate, 

ODDR&E 

SESSION 

CHAIR 
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9:10  AM 

8853  -  C4I  Architecture 
for  Joint  AS  W 

Mr.  Gregory  Miller, 

Naval  Postgraduate 

School 

9065  -  Rapidly 
Implementing  Lean 
CMMI®  Processes  That 
Meet  Business  Needs 

Mr.  Tim  Olson,  Lean 
Solutions  Institute,  Inc 

8784  -  Establishing 
a  Departmental- 
Level  Systems-of- 
Systems  Engineering 
Management  Construct 
for  the  Department 
of  the  Navy,  Progress 
Report 

Mr.  John  Kevin  Smith, 
Asst  Sec  of  the  Navy  for 
RD&A,  Chief  Engineer 

8829  "  The  Army 

Health  Hazard 

Assessment  Program’s 

Medical  Cost  Avoidance 

Model 

Dr.  Timothy  A. 

Kluchinsky,  Department 

of  Army 

8:35  AM 

8788  -  Data  sharing  in 
a  Stability  Operations 
Community  of  Interest: 
Utilizing  a  pilot  program 
to  prove  concepts  and 
develop  trust. 

Mr.  Gerald  Christman, 
Femme  Comp  Inc. 

9034  -  Sustainment 
and  Continued 
Institutionalization 
of  Best  Practices  and 
CMMI®  at  SPAWAR 

Mr.  Michael  Kutch, 
SPAWAR  Systems 

Center  Atlantic 

8960  -  A  Distillation  of 
Lessons  Learned  from 
Complex  System  of 
Systems  Acquisitions 

Dr.  Richard  Turner, 

Stevens  Institute 

9042  -  Bounding  the 
Human  Within  the 

System 

Mr.  Michael  Mueller, 

U.S.  Air  Force  Center 

for  Systems  Engineering 

8:00  AM 

8780  -  Net-Centric  Best 

Practices 

Mr.  Hiekeun  Ko,  JPEO- 
CBD  -  Software  Support 
Activity 

9003  -  CMMI®  for 
Executives 

Mr.  Geoff  Draper,  Harris 
Corporation 

NDIA  SoS  Committee 
Report 

Dr.  Judith  Dahman, 
Systems  Engineering 
Directorate,  ODDR&E/ 
MITRE 

8975  -  What  is  Human 
Systems  Integration 
(HSI)  and  why  should 
we  do  it? 

Mr.  Stuart  Booth, 

Systems  Engineering 
Directorate,  ODDR&E 

SESSION 

CHAIR 
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8891  "A  comprehensive 

overview  of  techniques 

for  measuring  system 

readiness 

Mr.  James  Bilbro,  JB 

Consulting  International 

9026  -  Early  SE 

Determination  of  Best- 

Fit  System  Life  Cycle 

Processes 

Dr.  Barry  Boehm,  USC 

8839  -  Navy  Systems 

Engineering  Technical 

Review  Process 

Ms.  Melinda  Reed,  DoD 

(ASN  RDA  CHSENG) 

8901  -  Review  Results 

of  the  NDIA/OSD 

Software  Test  Summit/ 

Workshop 

Mr.  Thomas  Wissink, 

Lockheed  Martin 

IS&GS 

8833  "  Communicating 
Risk:  Air  Force  RI3 
Methodology 

Mr.  John  Cargill,  AF 

Cost  Analysis  Agency 

8813  -  Emerging  Roles 

for  Systems  Engineering 

in  Defense  Decision 

Making;  Better  Aligning 

Requirements  and 

Acquisition  with  the 

Budget  and  Security 

Environments 

Mr.  Vincent  Roske, 

Institute  for  Defense 

Analyses 

8823  -  Win  and 

Influence  Design 

Engineers — Change 

Their  Affordability  DNA 

Mr.  Tim  Morrill, 

Raytheon  Company 

8814  -  Joint  Mission 
Environment  Test 
Capability  (JMETC), 
Lowering  technical 

Risk  by  Improving 
Distributed  Test 
Capabilities 

Mr.  Chip  Ferguson, 
JMETC 

8894  -  Air  Force 

Initiative  —  High 
Confidence  Technology 
Transition  Planning 
Through  the  Use  of 
Stage-Gates  —  Update 

Mr.  Randy  Bullard, 

U.S.  Air  Force  Materiel 
Command 

8949  -  Updated  DoD 
5000  and  CJCS  3170 
Policies:  A  Requirements 

to  Acquisition  Gap 

Analysis 

Mr.  John  Lohse, 

Raytheon  Company 

8863  -  Using 

Requirements 

Compliance  to  Identify 

Gaps  Between  the 

Technical  Solution  and 

Requirements 

Mr.  Frank  Salvatore, 

High  Performance 

Technologies,  Inc. 
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8883  -  Test  & 
Evaluation  Products 
for  the  Systems 
Engineering  Reviews 

Mr.  Woody  Eischens, 

OUSD(AT&L)/ 

DDR&E/DT&E 

8900  -  DOD’s 

Weapon  System 
Portfolio:  Are  Results 
Getting  Any  Better? 

Mr.  Michael  Sullivan, 
U.S.  Government 
Accountability  Office 

Q&A:  8924,  8925 , 

8933-  Early  Systems 
Engineering  in  DoDI 
5000.02 

Dr.  Judith  Dahmann, 

Ms.  Lisa  Reuss, 

Systems  Engineering 
Directorate,  ODDR&E 

8974  -  Transforming 
Systems  and  Software 
Engineering  Across  an 
Enterprise 

Mr.  Jeffery  Wilcox, 
Lockheed  Martin 
Corporation 

8882  -  Test  &  Evaluation 
Strategy  for  the 

Technology  Development 
Phase 

Ms.  Darlene  Mosser- 
Kerner,  OUSD(AT&L)/ 
DDR&E/DT&E 

8963  "  Air  Force  Concept 
Maturity  Assessment 

Mr.  George  Freeman, 

U.S.  Air  Force,  Center  for 
Systems  Engineering 

Panel  Topic:  8924,  8925 , 8933  -  Early  Systems 

Engineering  in  DoDI  5000.02 

Dr.  Judith  Dahmann,  Ms.  Lisa  Reuss,  Systems 

Engineering  Directorate,  ODDR&E 

8990  -  Systems  Engineering 
for  Rapid  Capability 
Development 

Mr.  Thomas  McDermott, 
Georgia  Tech  Research 
Institute 

8848  -  Integrated 

Testing:  We  Can  Do  It 

Dr.  Beth  Wilson, 

Raytheon  Company 

8916  -  System  Readiness 
-  Assessing  Technical  Risk 
Throughout  the  Lifecycle 

Mr.  James  Thompson, 
Systems  Engineering 
Directorate,  ODDR&E 

8816  -  Mind  the  GAPs-a 
Systems  Engineering 
Implementation  of  DoDI 
5000.02 

Dr.  Thomas  Christian, 

U.  S.  Air  Force 
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8973  -  C- 17  Transition  to 

Criteria-based  Airworthiness 

Certification 

Mr.  Christian  Stillings,  USAF 

516AESG 

8879  -  TRL  Vectors  in  IPPD- 

based  Portfolio  Management 

Mr.  Michael  Bartmess, 

General  Dynamics/AIS 

8795  -  Mission-based  Test 

and  Evaluation  Strategy: 

Creating  Linkages  between 

Technology  Development  and 

Mission  Capability 

Mr.  John  Beilfuss,  U.S.  Army 

Research  Laboratory 

8948  -  Value  Engineering 

Applications  in  Service 

Contracts 

Dr.  Jay  Mandelbaum,  Value 

Engineering  Applications  in 

Service  Contracts 

8855  "  Certify  and  Fly 
Right:  Preparing  for  DO- 
297  Certification 

Mr.  Ketih  Custer, 

Esterline  Control 

Systems- AVISTA 

8870  -  S&T  Portfolio 
Maturity  &  Performance 
Analysis:  The  Concept 

of  Critical  Research 

Elements 

Mr.  Has  Patel,  Infologic, 

Inc. 

8835  -  T&E 

Collaboration  and 

Contributions  during 

Early  Program 

Acquisition 

Mr.  Stephen  Scukanec, 

Northrop  Grumman 

Corporation  Aerospace 

Systems 

9414  -  Correcting 

Deficiencies  in  the 

Systems  Engineering  of 

Tactical  Weapons 

Mr.  Marvin  Ebbert, 

Raytheon  Missile 

Systems 

9014  -  SAVI:  Aerospace 
Platform  Development 
and  Certification  Using 
Modeling  and  Simulation 
to  “Integrate,  then  Build” 

Mr.  Gregory  Pollari, 

Rockwell  Collins 

8798  -  The  New 
Technology  Readiness 

Assessment  Process 

Dr.  Jay  Mandelbaum, 

Institute  for  Defense 

Analyses 

9082  -  Including 
Environment,  Safety,  and 

Occupational  Health 

(ESOH)  Requirements 

in  Joint  Capabilities 

Integration  and 

Development  System 

(JCIDS)  Documents 

Mr.  Sherman  Forbes, 

U.S.  Air  Force 

8902  -  Systems 

Engineering  Leading 

Indicators:  Insight 

into  Effective  Systems 

Engineering 

Mr.  Gary  Roedler, 

Lockheed  Martin 

Corporation 
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8935  -  Systems  of 

Systems  Systems 
Engineering  and  Test  and 
Evaluation 

Dr.  Judith  Dahmann, 
Systems  Engineering 
Directorate,  ODDR&E/ 

MITRE 

8770  -  Incorporating 
Maturity  Assessment 
into  House  of  Quality 
for  Improved  Decision 
Support  Analysis  and 

Risk  Management 

Mr.  Pavel  Fomin,  U.S. 

Air  Force 

9016  -  A  Framework  for 
Enhancing  Forward- 
looking  Capability 
Delivery  Metrics 

Mr.  Leonard  Sadauskas, 

DoD  CIO  CT&S 

8847  -  Tailoring  the  SE 
Process  to  Effectively 
Complement  the  SW 

Agile  Development 

Process 

Mr.  William  Lyders, 

ASSETT  Inc. 

8849  -  Joint  Integration  and 
Interoperability  Lab  (JSIIL) 

Mr.  Steven  Whitehead, 

SL,  J8  Technical  Director, 
USJFCOM 

9017  -  Linking  Systems 
Engineering  Artifacts  with 
Complex  Systems  Maturity 

Assessments 

Dr.  Brian  Sauser,  Stevens 
Institute  of  Technology 

8846  -  Air  Force  Materiel 

Command  Early  Systems 
Engineering 

Dr.  Brian  Kowal,  USAF 

8893  -  Rapid  Development 

Mr.  Michael  Gaydar, 

NAVAIR 

8825  -  Test  and  Evaluation 
in  a  System  of  Systems 
Environment 

Mr.  Edwin  McDermott,  653 
ELSW,  Electronic  Systems 
Center 

899 1  -  Systems  Engineering 
for  the  Science  &  Technology 
Community 

Mr.  Russell  Menko,  U.S. 

Army  RDECOM/TARDEC 

8951  -  USAF  View  of  NRC 
“Pre-A  Systems  Engineering” 
Study  Committee 

Recommendations  As 

Addressed  By  Levin-McCain 
(P.L.  111-23;  “Weapon 

Systems  Acquisition  Reform 

Act  of  2009”) 

Mr.  Jeff  Loren,  SAF/AQR 
(Alion  Science  &  Technology) 

8851  -  Rapid  Development 
and  Integration  of  Remote 
Weapon  Systems  to  Meet 
Operational  Requirements 

Mr.  Joseph  Burkart,  NSWC 
Crane,  Small  Arms  Air 

Platform  Integration 
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THURSDAY,  OCTOBER  29,  CONCURRENT  SESSIONS 


11:25  AM 

8996  -  Engineering 

Improvement  in 

Software  Assurance:  A 

Landscape  Framework 

Ms.  Lisa  Brownsword, 

SEI 

8789  "  Achieving 

Acquisition  Excellence 

via  Improving  the 

Systems-Engineering 

Workforce 

Dr.  Kenneth  Nidiffer, 

SEI 

8915  -  System  of 

Systems  Challenges  and 

Solutions:  Case  Study 

Insights 

Mr.  John  Colombi, 

U.S.  Air  Force  Institute 

of  Technology 

8844  -  The  Power 

of  the  Spec: 

Understanding 

the  Many  Diverse 

Roles  in  SE  of  Good 

Specifications  & 

Standards.” 

Mr.  Robert  Kuhnen, 

U.S.  Air  Force 

10:50  AM 

9104  -  Static  Code 
Analysis:  Best  Practices 
for  Software  Assurance 

in  the  Acquisition  Life 

Cycle 

Mr.  Paul  Croll,  CSC 

9088  -  Enhancing 

Systems  Engineering 

Competencies  in  the 

Enterprise 

Mr.  Gary  Roedler, 

Lockheed  Martin 

Corporation 

8903  -  Global  Earth 

Observation  System  of 

Systems  (GEOSS) 

Mr.  Lawrence 

McGovern,  Northrop 

Grumman  Electronic 

Systems 

8922 -The  Role  of  DoD 

in  Systems  Engineering 

Standards  and  Models 

Mr.  Donald  Gantzer, 

Systems  Engineering 

Directorate,  ODDR&E 

10:15  AM 

8812  -  A  Systems 
Engineering  Approach 
to  Multi-Level  Security 
in  a  Service  Oriented 
Architecture 

Mr.  Timothy  Greer, 
Lockheed  Martin 
Corporation 

8966  -  Improving 

Systems  Engineering 

Curriculum  Using  a 

Competency-Based 

Assessment  Approach 

Ms.  Alice  Squires, 

Stevens  Institute  of 

Technology 

9024  -  Dynamic 

Modeling  of 

Programmatic  and 

Systematic 

Dr.  Brian  Sauser, 

Purdue  University 

8945  -  Standards 

Based  Development 

Environment 

Mr.  Christopher  Oster, 

Lockheed  Martin 

Corporation 
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9:10  AM 

8739  -  Quality  Assessment 
of  Software-Intensive 

System  Architectures 
and  their  Requirements 
(QUASAR) 

Mr.  Donald  Firesmith, 

SEI 

8943  -  Team  SE  Skill  Set 

Mr.  Charles  Garland, 

U.S.  Air  Force  Center  for 

Systems  Engineering 

8969  -  An  Introduction 
to  Influence  Maps: 
Foundations, 

Construction,  and  Use 

Mr.  James  Smith,  SEI 

Process  management  and 

tool  selection  to  minimize 

risk  of  hand-arm  vibration 

syndrome 

Mr.  Sherman  Forbes,  U.S. 

Air  Force 

8:35  AM 

8820  -  Graduate  Software 
Engineering  Reference 
Curriculum  (GSwERC) 

Ms.  Nicole  Hutchison, 
Analytic  Services,  Inc. 

9076  -  Assessing  Systems 
Engineering  Personnel 
Competency:  Framework 
and  Tool  Experience 

Dr.  Barry  Boehm, 

University  of  Southern 
California 

8964  -  Software  Assurance 
in  a  System  of  Systems 
World:  Interoperability 
Challenges  -  Reports  from 

the  Field 

Dr.  Carol  Sledge,  SEI 

9064  -  Economics 
of  Human  Systems 
Integration:  Early  Life 

Cycle  Cost  Estimation 

Using  HSI  Requirements 

2ndLt  Kevin  Liu,  USMC, 

MIT 

8:00  AM 

8977  _  Overview  of  DoD 
Software  Engineering 

Initiatives 

Mr.  Scott  Lucero,  Systems 
Engineering  Directorate, 

ODDR&E 

8926  -  Systems  Engineering 
Workforce  Development 
Update 

Dr.  Don  Gelosh,  Systems 
Engineering  Directorate, 

ODDR&E 

9083  -  Requirements 
Engineering  for  Systems  of 
Systems 

Mr.  Soumya  Simanta,  SEI 

8937  -  Integrating  the 

Human  into  the  system, 
integrating  HSI  Tools  into 
Systems  Engineering 

Dr.  Jennifer  Narkevicius, 
Jenius  LLC 
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9054  -  A- 10  Avionics 
System  Architecture 
Trade  Study  and 

Analysis  (AVSATA) 
Program 

Mr.  Richard 

Sorensen,  KIHO 

Military  Acquisition 

Consulting,  Inc. 

8947  _  Report  on  a 

Study  on  Management 

Concepts  for  Broadly- 

Needed  Modeling  and 

Simulation  Tools  in  the 

U.S.  Department  of 

Defense 

Dr.  James  Coolahan, 

Johns  Hopkins 

University/APL 

9008  -  Upgrade  Fluid 

System  Filter  Element 

Monitoring  To  Increase 

Operational  Reliability 

and  Support  Condition 

Based  Maintenance 

Capability 

Mr.  Gary  Rosenberg, 

Constellation 

Technology 

Corportation 

8824  -  Delivering 

DoDAF  Version  2.0  to 

Architects  and  Systems 

engineers  for  IT 

Systems  and  Services 

Mr.  Walt  Okon, 

Department  of  Defense 

CIO,  Enterprise 

Architecture 

8946  -  Protecting  the 
Mission,  Preserving 

Legacy  and  Promoting 

Growth 

Ms.  Patti  Scaramuzzo, 

Lockheed  Martin 

Corporation 

9052  -  Best  Practices 
in  Contracting  for 

Models,  Simulations,  and 

Associated  Data 

Mr.  Dennis  Shea,  CNA 

9039  -  Applying  Systems 

Engineering  to  Fielded 

Weapon  Systems  and 

End-Items 

Mr.  Michael  Ucchino, 

AF  Center  for  Systems 

Engineering 

8830  -  Systems 

Engineering  Needs  of  the 

DoDAF  -  Report  of  the 

Architecture  Frameworks 

Working  Group 

Mr.  Joe  Kuncel, 

Northrop  Grumman 

Corporation 

8880  -  Naval 

Postgraduate  School 
Advanced  Seabase 

Enabler  Project:  A 

Systems  Engineering 

Case  Study 

Mr.  Lance  Flitter, 

NSWC,  Carderock 

Division 

8759  -  A  Systems 
Engineering  Framework 
for  Integrating  M&S 
Development  Best 

Practices 

Dr.  Katherine  Morse, 

Johns  Hopkins 

University/APL 

8988  -  How  to  Save 

Time  and  Money  Using 

Lean  Maintenance 

Processes 

Mr.  Tim  Olson,  Lean 

Solutions  Institute,  Inc. 

8831  -  Human- Centered 

Design  in  Systems 

Engineering:  Human 

View  Methodology 

Dr.  Robert  Smillie, 

SPAWAR 

'3UJ 

‘SdlSojOUCfOdJ^  ‘UOSJ3J3J  BUBQ  'JJ/\[ 

TdV/AHf  ‘uBqBtoo3  uixf  jjv 

Z LV'OSO  ‘3uoduiBiS 
Xuoqjuy  ujy  puB  /Cuvdtuoj 
uoac/iiCptf  ‘qajiAjoojy  pof  -jjy 

uoupj.odj.o'j 

upmmnip)  doiquoj\j  ‘pounyj  aof  ujy 

J  UOISSIjy 

SuuaauiSmj  suiajsdq  popoBjj 

L^DVUL 

J  AVSIA^Bg 

uopBjnuns  Suippojy 
£^D\XIX 

J  J  M31aXbQ 

suiajsXq  sopsiSoq 

Z^lDYdX 

J  JJ  AY3TaXb£[ 
ajnjoajiqojy 

I^DVUL 

9007  -  Technology 
Maturation  for  the 

Automated  Aerial 

Refueling  (AAR)  Project 

Ms.  Carol  Ventresca, 
SynGenics  Corporation 

9048  -  Revisions  to  the 
Acquisition  Modeling  & 

Simulation  Master  Plan 

Mr.  Stephen  Swenson, 
Systems  Engineering 
Directorate,  ODDR&E 

8908  -  Successful  First 
AESA  Deployment 
through  Application  of 
System  Engineering 

Mr.  Scott  Nichols, 

Raytheon  Company 

9097  -  Acquisition  ESOH 
Risk  Management-How  to 

Make  It  Work 

Mr.  Bob  Smith,  Booz 

Allen  Hamilton 

8980  -  Using  Model- 
driven  Engineering 
Techniques  for  Integrated 
Flight  Simulation 
Development 

Mr.  Douglas  Fiehler, 
Raytheon  Missile  Systems 

8950  -  Live,  Virtual, 

Constructive  Architecture 

Roadmap:  The  Quest  for 
Interoperability,  Standards, 

and  Reuse 

Dr.  Gary  Allen,  Joint 
Training  Integration  & 

Evaluation  Center 

9031  -  Supportability 

Lessons  Learned  with  Line 

Replaceable  Modules 

Ms.  Heity  Hsiung, 

Raytheon  Company 

8920  -  A  Methodology 
for  Assessing  Systems 
Engineering  Practices 

Ms.  Lauren  Levy,  Johns 
Hopkins  University/APL 

8875  -  Tomahawk  Weapon 
System  Development  and 
Integration 

Mr.  Gustavo  Rivera,  Naval 
Surface  Warfare  Center, 
Dahglren  Division 

8939  -  Understanding  the 

New  DoD  Instruction 

5000.61:  “DoD  Modeling 
&  Simulation  Verification, 

Validation  and  Accreditation 

(W&A)” 

Mr.  Michael  Truelove, 

Systems  Engineering 
Directorate,  ODDR&E 

9063  -  An  Integrated  RAM 
Approach  to  System  Design 
and  Support 

Mr.  Robert  Finlayson,  Johns 
Hopkins  University/APL 

8887  -  Achieving  a  Systems 
Engineering  Culture  in  a 
Science  and  Technology 
Laboratory  Environment 

Mr.  Robert  Rapson, 

Materials  and 

Manufacturing  Directorate, 

AFRL 
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8802  -  Open  Source  Technology 
for  Enterprise  Health 

Management 

8901  -  Review  Results  of  the 

NDIA/OSD  Software  Test 

Summit/Workshop 

9506  -  Software  Acquisition 
Management  Practical  Experience 
Mr.  James  Jones,  SSAI 

spa 

O 

CO 

^  U  s  -h  9 

43  Q  8  |  Q  o 

s  °  00  K 

Mr.  Edward  Beck,  CSC 

Mr.  Thomas  Wissink,  Lockheed 

Martin  IS&GS 

0000  -  Implementing  CMMI  on  a 
COTS  Modification  Effort 

Mr.  Dave  Castellano,  U.S.  Army 

TRACK 7 

Work  Force  Development 
Palm  I 

Dr.  Don  Gelosh,  Systems 
Engineering  Directorate, 
ODDR&E 

and  Mr.  Mike  Ucchino, 

U.S.  Air  Force  Center  for 
Systems  Engineering 

8956  -  Systems  Engineering 
Approach  to  Workforce 
Development 

Mr.  James  Miller,  U.S.  Air  Force 

9046  -  Developing  an 

Introductory  Systems  Engineering 
Practitioners  Course:  “Model- 
Based  Systems  Engineering 
(MBSE)  With  SysML” 

Mr.  Joseph  Wolfrom,  Johns 
Hopkins  University/ APL 

8878  -  Advanced  Simulation 

Course  for  Army  Simulation 
Management  Professionals 

Dr.  Gene  Paulo,  Naval 

Postgraduate  School 

TRACK 6 
Enterprise  Health 
Management 
Mission  III 

Mr.  Howard  Savage, 

Savage  Consulting  and 

Mr.  Chris  Reisig,  The 

Boeing  Company 

8815  -  Applying  Systems  Engi¬ 
neering  to  Operational  System 
Improvements 

Ms.  Ryanne  Gentry,  Acquisition 
Logistics  Engineering 

8842  -  Applications  in  Integrated 
Diagnostics 

Mr.  Jimmy  Simmons,  Georgia 

Tech  Research  Institute 

8884  -  Tactical  Wheeled  Vehicle 
Integrated  Diagnostics 

Mr  Lawrence  Osentoski,  DRIVE 
Developments,  Inc. 

TRACK  5 

Systems  Engineering 
Development 
Environment 

Mission  II 

Mr.  A1  Brown,  The 

Boeing  Company 

8967  _  Generating  Visual  and 
Interactive  Output  from  System 
Engineering  Tools 

Mr.  John  Schatz,  Systems  and 
Proposal  Engineering  Company 

9015  -  Challenges  and  Benefits  of 
applying  ISO  STEP 

Mr.  Stuart  Booth,  Systems 
Engineering  Directorate, 

ODDR&E 

9059  "  Smallsat  Conceptual 

Design  Trade  and  Cost  Modeling 

Tool 

Dr.  Deganit  Armon,  Advatech 
Pacific,  Inc 

TRACK 4 
Practical  Systems 
Engineering 
Mission  I 

Mr.  Dana  Peterson, 

DRS  Technologies,  Inc. 

8976  -  A  Systems  Engineering 
Model  for  Roadmap  Alignment 

Mr.  Si  Dok,  U.  S.  Army 

TARDEC 

9080  -  Rapid  Systems  Engineering 

of  the  MRAP  Gunner  Restraint 

System  Saves  Lives 

Ms.  Michelle  Bowen,  JPO  MRAP 
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AWARD  INFORMATION 

2009  LT  GEN  THOMAS  R.  FERGUSON,  JR. 

SYSTEMS  ENGINEERING  EXCELLENCE  AWARD 

The  National  Defense  Industrial  Associations  Systems  Engineering  Excellence  Awards  were  established  in 
2003  to  honor  the  memory  of  Lt  Gen  Thomas  R.  Ferguson,  Jr.,  USAF,  whose  leadership  embodied  the 
highest  ideals  in  Defense  Systems  development  and  deployment. 

The  awards  are  given  to  an  individual  and  to  a  group  demonstrating  outstanding  achievement  in  the 
practical  application  of  Systems  Engineering  principles,  promotion  of  robust  systems  engineering  principles 
throughout  the  organization,  or  effective  systems  engineering  process  development  during  the  previous 
year.  Their  systems  engineering  contributions  should  have  demonstrably  helped  achieve  significant  cost 
savings  due  to  new  or  enhanced  processes  procedures  and/or  concepts,  increased  mission  capabilities,  or 
substantially  increased  performance.  The  2009  awardees  are: 


►  Systems  Engineering  Individual  Leadership  Award:  Mr.  Brian  Wells 

►  Systems  Engineering  Group  Award:  Center  for  Advanced  Life  Cycle  Engineering 


PAST  AWARD  WINNERS: 


2003: 

►  Systems 
2004: 

►  Systems 
2005: 

►  Systems 
2006: 

►  Systems 

►  Systems 

►  Systems 
2007: 

►  Systems 

►  Systems 
2008: 

►  Systems 

►  Systems 


Engineering  Individual  Leadership  Award:  Mr.  Robert  Rassa 

Engineering  Individual  Leadership  Award:  Honorable  Mike  Wynne 

Engineering  Individual  Leadership  Award:  Mr.  Mark  Schaeffer 

Engineering  Individual  Leadership  Award:  Mr.  Kelly  Miller 

Engineering  Individual  Practitioner  Award:  Mr.  David  Strimling 

Engineering  Group  Award:  NUWC  Division  Newport  Critical  Transducer  Program  Staff 

Engineering  Individual  Leadership  Award:  Mr.  Robert  Skalamera 
Engineering  Group  Award:  Submarine  Warfare  Federated  Tactical  System  Team 

Engineering  Individual  Leadership  Award:  Honorable  James  Finley 

Engineering  Group  Award:  Tactical  Direction  Agent  Team  for  LCS  Mission  Package  Project 
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AWARD  INFORMATION 

DEPARTMENT  OF  DEFENSE  AND  THE  NATIONAL  DEFENSE  INDUSTRIAL  ASSOCIATION 
2008  TOP  5  DEPARTMENT  OF  DEFENSE  PROGRAM  AWARDS 

The  Department  of  Defense  Executive  Agent  for  Systems  Engineering  and  the  Systems  Engineering  Division  of 
the  National  Defense  Industrial  Association  are  pleased  to  announce  the  selections  of  the  2008  Top  5  Department 
of  Defense  Program  Awards.  The  2008  Program  awardees  are: 

►  Wideband  Global  SAT COM:  U.S.  Air  Force  PM;  Boeing  Company  Space  &  Intelligence  Systems  Group 

►  Joint  Light  Tactical  Vehicle:  U.S.  Army/USMC  PMs;  BAE  Systems  Land  &  Armaments;  General  Tactical 
Vehicles;  Lockheed  Martin  Systems  Integration 

►  STRYKER  Modernization:  U.S.  Army  PM;  General  Dynamics  Land  Systems 

►  Broad  Area  Maritime  Surveillance  Unmanned  Aircraft:  U.S.  Navy  PM;  Northrop  Grumman  Corporation 

►  Aviation  Maintenance  Training  Continuum  System:  U.S.  Navy  PM;  Raytheon  Company;  Paladin  Data 
Systems  Corporation 

The  Awards  are  presented  to  both  the  DoD  project  office  and  the  industry  prime  contractor  in  recognition  of 
total  program  performance  in  a  DoD/industry  team  effort. 

PAST  AWARD  WINNERS: 

2005  Top  5  Department  of  Defense  Programs: 

►  Centaur 

►  Integrated  Exploitation  Capability 

►  P-8A  Multi  Mission  Maritime  Aircraft 

►  Mission  INtegration  &  Development 

►  Tomahawk  Weapons  System  Program  PMA-280 

2006  Top  5  Department  of  Defense  Programs: 

►  Advanced  Extremely  High  Frequency  Mission  Control  System 

►  Advanced  Field  Artillery  Tactical  Data  System 

►  DDG  1000  MK57  Vertical  Landing  System 

►  Portable  Excalibur  ECS 

2007  Top  5  Department  of  Defense  Programs: 

►  Effects  Management  Tool 

►  MH-60  R/S  Link  16 

►  Mortar  Fire  Control  System  -  Dismounted 


SYSTEMS  ENGINEERING  CONFERENCE 
PROMOTIONAL  PARTNERSHIP 


THANK  YOU  TO  OUR  PROMOTIONAL  PARTNERS: 


PTC  provides  product  lifecycle  management  solutions  designed  to  meet  the 
requirements  of  the  global  aerospace  and  defense  industry.  These  solutions 
enable  digital  automation  of  product  development  and  program  management 
processes,  complete  visibility  and  control  over  program  information  for  secure, 
collaborative  product  development  as  well  as  dynamic  publishing  that  allows  you  to  produce  vital  service  information  directly 
in  the  standards-based  formats  -  either  in  print  or  on  the  Web.  PTC  is  an  industry  leader,  serving  the  product  development 
needs  of  the  top  20  A&D  companies.  Further  information  is  available  at  http://www.ptc.com/go/a-d. 


^University  of  Phoenix® 

*\T 


At  University  of  Phoenix,  we've  been  thinking  ahead  for  more  than  30 
years.  In  fact,  we  were  founded  in  1976  on  an  innovative  idea:  make 
higher  education  highly  accessible  for  working  students. 

Still  guided  by  this  idea,  University  of  Phoenix  has  helped  transform  the 
landscape  of  higher  education  in  widely  recognized  ways. 


Many  of  the  conveniences  that  21st-century  students  now  take  for  granted — evening  classes,  flexible  scheduling,  continuous 
enrollment,  a  student-centered  environment,  practitioner  faculty,  online  classes,  online  library,  ebooks,  computer  simulations — 
were  pioneered  or  made  acceptable  through  University  of  Phoenix's  efforts. 

Configuration  Management  Data  Management  Coursework 


This  program  exposes  students  to  the  most  important  principles  concerning  configuration  management  history,  configuration 
identification,  configuration  change  management,  and  data  management.  Courses  are  available  over  the  internet  through  our 
Online  Learning  System  (OLS)  or,  in  small  classes  at  select  classroom  locations  as  available. 


To  learn  more  contact  University  of  Phoenix  -  Center  for  Professional  Development  at  1  (800)  325-1509  or  via  email 
prodev@phoenix.edu. 


SYSTEMS  ENGINEERING  CONFERENCE 
PROMOTIONAL  PARTNERSHIP 


Lean  Solutions  Institute,  Inc.  (LSI)  specializes  in 

LEAN  SOLUTIONS  INSTITUTE,  Inc.  helping  organizations  to  rapidly  achieve  measur- 

LEAN  SOLUTIONS™  FOR  TOUR  ORGANIZATION  fb'e  results  by  using  benchmarking  and  Lean 

SolutionsTM  (e.g.,  best  practices  to  implement 
CMMI®  in  a  lean  way)  to  successfully  improve 
client  products  and  services.  LSI  helps  organiza¬ 
tions  to  measurably: 

•  Achieve  ROI  (e.g.,  7:1) 

•  Increase  productivity,  performance  and  quality 

•  Reduce  cycle  time/schedule 

•  Reduce  defects  (e.g.,  post-release  defects),  rework  and  costs  of  poor  quality 

•  Achieve  world-class  results  (e.g.,  70-90%  defect  removal  efficiency  or  defects  removed  before  test) 

Systems  engineering  and  software  engineering  have  become  more  and  more  complex  over  the  years.  With  this  growing 
complexity,  processes  and  procedures  have  become  larger  and  more  complex.  Based  on  surveys,  most  organizations  do  not 
like  their  processes  and  procedures  (e.g.,  including  CMMI®  Maturity  Level  3-5  organizations)  and  they  can  have  some  of  the 
following  lean  problems: 

•  Too  large  and  complex  (i.e.,  not  lean  or  agile) 

•  Have  non-value  added  activities 

•  Lack  of  visualization  (e.g.,  pictures,  diagrams,  tables,  charts,  etc.) 

•  Difficult  to  use  (e.g.,  poor  usability) 

•  Lack  of  “chunking”  which  is  a  best  practice  for  usability  (7  plus  or  minus  2  principle) 

•  Lack  of  innovation 

•  Lack  of  “good  metrics”,  not  the  right  metrics,  or  not  lean  metrics 

LSI  has  a  patent  pending  approach  for  defining  systems  engineering  and  software  engineering  processes  (e.g.,  CMMI®  com¬ 
pliant  processes)  in  a  lean  (e.g.,  short,  usable,  visual)  way.  Although  this  approach  can  be  simple,  it  also  scales  up  to  handle 
complex  processes  (e.g.,  NASA  processes).  LSI  uses  “good  diagrams”  (i.e.,  process  models)  for  putting  the  5  Ws  (who,  what, 
where,  when,  why)  on  one  page.  These  visual  one-page  diagrams  along  with  a  page  of  support  text  typically  replace  about  25- 
30  pages  of  text.  For  example,  lean  CMMI®  processes  are  typically  about  20-25%  of  the  size  of  a  typical  CMMI®  implemen¬ 
tation,  and  take  half  the  time  to  implement  (e.g.,  1  year).  In  several  CMMI®  success  stories  (independently  verified)  using  the 
LSI  approach,  organizations  estimate  that  processes  are  about  20%  of  the  size  of  sister  business  units  with  a  similar  CMMI® 
rated  processes,  and  have  achieved  CMMI  maturity  levels  half  the  time  (or  less). 

LSI  can  help  your  organization  achieve  measurable  results,  reduce  size  and  complexity,  and  improve  processes  and  metrics 
to  become  much  more  lean,  “value  added”,  visual,  and  usable.  LSI  also  uses  an  ISO/Baldrige  approach  to  implementing 
CMMI®.  LSI  only  does  improvement  and  uses  independent  Authorized  SEI  Lead  Appraisers  to  objectively  verify  LSI  Lean 
SolutionsTM  for  CMMI®. 


CMMI  is  a  registered  trademark  of  the  U.S.  Patent  and  Trademark  Office  by  Carnegie  Mellon  University. 
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Major  Andrew  Gepp, 
USMC 

9014 

SAVI:  Aerospace  Platform  Development  and  Certification 

Using  Modeling  and  Simulation  to  “Integrate,  then  Build” 

Dr.  Don  Ward 

9015 

Challenges  and  Benefits  of  applying  ISO  STEP 

Mr.  Charlie  Stirk 

9017 

Linking  Systems  Engineering  Artifacts 
with  Complex  Systems  Maturity  Assessments 

Mr.  Kenneth  Michaud 

Mr.  Richard  Volkert 

Mr.  Eric  Forbes 

Dr.  Joes  Ramirez-Marquez 

9021 

Critical  Success  Factors  for  Milestone  Review  Risk  Identification 

Dr.  Jo  Ann  Lane 

9023 

Department  of  Energy  Office  of  Environmental  Management  s 

Technology  Readiness  Assessment  (TRA) /Technology  Maturation  Plan 
(TMP)  Process  Guide  and  Plans  for  TRA  Training 

Dr.  Stevem  Krahn 

Mr.  Kurt  Gerdes 

Dr.  Herbert  Sutter 

9024 

Dynamic  Modeling  of  Programmatic  and  Systematic 

Interdependence  for  System  of  Systems  Acquisition 

Dr.  Brian  Sauser 

Dr.  Muharrem  Mane 

Mr.  Alex  Gorod 

9025 

Defining,  Assessing,  and  Improving  Architecture  Competence 

Len  Bass 

Paul  Clements 

Suzanne  Garcia 

Rick  Kazman 

9026 

Early  SE  Determination  of  Best-Fit  System  Life  Cycle  Processes 

Dr.  Jo  Ann  Lane 

9027 

Department  of  Energy  Office  of  Environmental 

Management  s  Technology  Readiness  Assessment  (TRA)  Process 

Mr.  Kurt  Gerdes 

Dr.  Steven  Krahn 

Dr.  Herbert  Sutter 

9031 

Supportability  Lessons  Learned  with  Line  Replaceable  Modules 

Mr.  Joel  Moorvitch 

9034 

SUSTAINMENT  AND  CONTINUED  INSTITUTIONALIZATION 

OF  BEST  PRACTICES  AND  CMMI®  AT  SPAWAR 

Mr.  Michael  Knox 

9035 

Enabling  Collaborative  Decision  Making:  A  Process  for  Integrating 

Novel  Systems  Engineering  Tools  and  Methods  for  Renewable  Energy  Portfolio 
Analysis 

Mr.  Thomas  McDermott 

9041 

On  Modeling  and  Simulation  Methods  for 

Capturing  Emergent  Behaviors  for  Systems  of  Systems 

Dr.  Tommer  Ender 

Dr.  Santiago  Ballestrini- 
Robinson 

9043 

Implementing  the  Materiel  Availability  KPP  in  DoD  acquisition 
programs — balancing  life-cycle  costs  with  warfighter  needs 

Mr.  Pete  Nolte 

Mr.  John  Quackenbush 

9046 

Developing  an  Introductory  Systems  Engineering  Practitioners  Course: 
“Model-Based  Systems  Engineering  (MBSE)  With  SysML” 

Mr.  Michael  Pafford 

9054 

A- 10  Avionics  System  Architecture  Trade  Study  and  Analysis  (AVSATA)  Program 

Mr.  Adam  Grimm 

Mr.  Jerry  Coates 

9059 

Smallsat  Conceptual  Design  Trade  and  Cost  Modeling  Tool 

Mr.  John  Carsten 

Mrs.  Dana  Sherrell 

Mr.  Mike  Paisner 

Mr.  Mark  Sutton 

9060 

Modeling  and  Simulation  Support  for  the  Systems  Engineering 
of  Systems  of  Systems  (short  title  “M&S  Support  for  SoS  SE”) 

Dr.  William  Asrat 

9064 

Economics  of  Human  Systems  Integration: 

Early  Life  Cycle  Cost  Estimation  Using  HSI  Requirements 

Dr.  Ricardo  Valerdi 

9065 

Rapidly  Implementing  Lean  CMMI  Processes  That  Meet  Business  Needs 

Mr.  Tim  Olson 

9072 

1/2  Day  Tutorial  -  Leveraging  the  Defense  Acquisition 

Program  Support  (DAPS)  Methodology  for  Program  Success 

Mr.  Peter  Nolte 

9076 

Assessing  Systems  Engineering  Personnel  Competency: 

Framework  and  Tool  Experience 

Mr.  Dan  Ingold 

Dr.  Paul  Componation 

Dr.  Richard  Turner 

Ingold  Dan 

9078 

Tutorial:  Organizational  Implications  of  Systems  of  Systems 

Ms.  Lisa  Brownsword 

Mr.  Patrick  Kirwan 

9080 

Rapid  Systems  Engineering  of  the  MRAP  Gunner  Restraint  System  Saves  Lives 

Mr.  Michael  Perricane 

9081 

Testing  in  Service-oriented  Environments 

Mr.  Edwin  Morris 

Mr.  Sriram  Balasubrama- 
niam 

9083 

Requirements  Engineering  for  Systems  of  Systems 

Mr.  Edwin  Morris 

Dr.  Dennis  Smith 

Grace  Lewis 

Mr.  Patrick  Place 

9094 

DoD  Green  Procurement  Program  Update  and  Path  Forward 

Ms.  Sandy  Ross 

Ms.  Christina  Graven 

9095 

Acquisition  ESOH  Risk  Management  and  HAZMAT  Management  Part  I: 
Hazardouse  Materials  Management  Plan 

Mr.  Sherman  Forbes 

9097 

Acquisition  ESOH  Risk  Management-How  to  Make  It  Work 

Mr.  Sherman  Forbes 

9103 

The  Economics  of  CMMI 

Mr.  Mike  Campo 

9104 

Static  Code  Analysis:  Best  Practices  for  Software  Assurance 
in  the  Acquisition  Life  Cycle 

Mr.  Paul  Croll 

Defense  Research  and  Engineering: 

The  Path  Ahead 

The  Honorable  Zachary  J.  Lemnios 
Director,  Defense  Research  and  Engineering 


NDIA  SE  Conference  Keynote  Address 
10/27/09  Page-1 


12th  Annual  NDIA  Systems  Engineering  Conference 

October  27,  2009 


UNCLASSIFIED 


Our  Guidance 


*  Defense  Budget  Recommendation  Statement 
Secretary  of  Defense  Robert  M.  Gates,  April  06,  2009 

-  reaffirm  our  commitment  to  take  care  of  the  all-volunteer  force 

-  rebalance  this  Department’s  programs 

-  institutionalize  and  enhance  our  capabilities  to  fight  the  wars  we 
are  in  today  and  the  scenarios  we  are  most  likely  to  face  in  the 
years  ahead 

-  provide  a  hedge  against  other  risks  and  contingencies 

-  fundamental  overhaul  of  our  approach  to  procurement, 
acquisition,  and  contracting 

•  Economic  Club  of  Chicago 

Secretary  of  Defense  Robert  M.  Gates,  July  16,  2009 

-  What  is  needed  is  a  portfolio  of  military  capabilities  with 
maximum  versatility  across  the  widest  possible  spectrum  of 
conflict 
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Weapon  Systems  Acquisition 
Reform  Act  of  2009 


•Establishes  Director,  Systems 
Engineering  (SE)  and  Director, 
Developmental  Test  &  Evaluation 
(DT&E)  as  principal  advisors  to  the 
SECDEF  and  the  USD(AT&L) 

•Mandates  documented  assessment 
and  competitive  prototyping 

•Strengthens  technical  analysis  of 
cost  and  schedule  breaches  during 
development 


President  Barack  Obama  signing  the  Weapons 
Systems  Acquisition  Reform  Act  in  the  Rose 
Garden  at  the  White  House  Friday,  May  22, 
2009. 


Official  White  House  Photo  by  Samantha  Appleton 
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DDR&E  Imperatives 


1.  Accelerate  delivery  of  technical  capabilities  to 
win  the  current  fight. 

2.  Prepare  for  an  uncertain  future. 

3.  Reduce  the  cost,  acquisition  time  and  risk  of 
our  major  defense  acquisition  programs. 

4.  Develop  world  class  science,  technology, 
engineering,  and  mathematics  capabilities  for 
the  DoD  and  the  Nation. 
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Defense  Program  Support 
within  the  AT&L  Organization 


UNDER  SECRETARY  OF  DEFENSE 
(ACQUISITION,  TECHNOLOGY  AND  LOGISTICS) 

Honorable  Dr.  Ashton  B.  Carter 


Defense  Program  Support 


DEPUTY  UNDER 

ATSD  NUCLEAR,  CHEMIC 

DIRECTOR,  DEFENSE  ^ 

^  DEPUTY  UNDER 

DEPUTY  UNDER 

DIRECTOR  OF 

SECRETARY  OF  DEFENSE 

V  BIOLOGICAL  DEFENSE 

RESEARCH  &  ENGINEERING 

\secretary  OF  DEFENSE 

SECRETARY  OF  DEFENSE 

OPERATIONAL 

(ACQUISITION  &  TECHNOLOGY) 

1  PROGRAMS  I 

■(LOGISTICS  &  MATERIEL 

( INSTALLATIONS 

ENERGY  PLANS  & 

(Acting) 

/  READINESS)  (Acting) 

&  ENVIRONMENT) 

PROGRAMS 

^  Mr.  Shay  Assad  ^ 

Honorable  Andrew  Weber  > 

Honorable  Zachary  Lemnios^ 

Mr.  Alan  Estevez 

Dr.  Dorothy  Robyn 

Proposed 

4  Deputy  Under  Secretary  Positions 


DIRECTOR, 
MISSILE  DEFENSE 
AGENCY 

LTG  Patrick  J.  O’Reilly 


DIRECTOR, 

TEST  RESOURCE 
MANAGEMENT  CENTER 

Dr.  John  Foulkes 
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DDR&E  Organization 


Joint  Support 

Knollmann 


Joint 

Interoperability 

Quinlan 


DTIC 

Ryan 


Political  appointee 
Career  SES  /  GS-15 


DMEA 

Glum 


Strategic  Cell 

Kurjanowicz 


Director,  Research 
Honey 


PD  -  van  Tilborg 


Technologies 

van  Tilborg 


STEM 

Adolfie  (Acting) 


Basic  Science 

Staffin 


Laboratories 

Fischer 


Systems  Analysis 

Baldwin 


Major  Program  Support 

Thompson 


Mission  Assurance 

Torelli 


Director,  Rapid  Fielding 
Wyatt 


PD  -  Riley 


Joint  Rapid 
Acquisition  Cell 

Dee 


Complex  Systems 

Perkins 


Rapid  Reaction 
Technology  Office 

Fogg  (Acting) 


Program  Oversight 

Di  Petto 


Program  Guidance  And 
Assessment 
_ TBD _ 
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New  Coordinates 


Innovation 

Speed 

Agility 
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Comments  from  COCOMs 


“We  need  to  detect  lEDs  at  range...  I  am  willing  to  test  technologies  in  the  field... 
We  need  persistent  communications  on  the  move...” 


“I  need  the  70%  solution  today,  rather  than  the  100%  solution  in  5-8  years...” 


“...we  are  concerned  about  our  technological  edge  against  a  near  peer 
competitor...” 


“It  took  us  10  years  to  get  to  the  Moon,  we  are  8  years  into  our  research  efforts  for 
defeating  IEDs...we  need  to  find  a  solution  to  reliably  detect  and  defeat  lEDs  at 
range...' 

“I  like  the  1-year  acquisition  cycle  rather  than  the  standard  5-  to  8-  year  cycle,  get 
the  prototypes  into  the  hands  of  the  warfighters,  turn  the  feedback  into  a  quick 
redesign  and  deliver  relevant  capability  now...” 

“Often  times  we  fail  due  to  shortage  of  imagination...” 
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Perspective 
for  the  Next  Decade 


1950  1960  1970  1980  1990  2000  2010  2020 


National 

Security 

Challenges 


Cold  War 


Vietnam  War 


▲ 

Desert  Storm 


▲ 

Collapse  of 
Soviet  Union 


Bosnia 


OEF 


Kosovo 


I  Irregular/Hybrid 
Warfare 


Defense 

Capabilities 


Satellite  comms  C4ISR 


ICBM 

LGB’s 


GPS 


Stealth 


Precision 

Strike 

UAV 

Robotics 


Nuclear  propulsion 


Night  Vision 


Human  Terrain 
Ubiquitous  Observation 
Contextual  Exploitation 
Scaleable  Action 


Enabling 

Technologies 


Transistor  Composite  Materials  MEMS 

Solid  state  laser  Superconductors 

Space  tracking  Web  MIMIC 

Digital  computing  IR  Sensors 

High  Performance  Computing 


•Advanced  Electronics, 
Photonics  Algorithms,  MEMS 

•  Nano;  Meta;  &  New  Materials 

•  Cognitive  Computing 

•  Bio-Revolution 
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Four  Key  Challenges 
to  our  Technical  Base 


Tech  Areas 


Time 


Impact 
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We  are  in  Competition 
for  the  Best  and  Brightest 


/43 


Tl 

OOR&f  ^  ^ 

-/  .  V 


^HrHrHrHrHrHrHT-Hr'jr'JfN 


— China 
— Japan 
— S.  Korea 
— U.S. 

— U.K. 

— Germany 


S&E  Indicators,  2008 
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The  STEM  Workforce  Challenge 


Future  Workforce 


70.00% 

60.00% 

50.00% 

40.00% 

30.00% 

20.00% 

10.00% 

0.00% 


lillenium 
(1990- 
pre 


Baby  Traditional 

Boomers  Generation 
(1946-1964)  (born  before 


□  National  Workforce 

■  DoD  Workforce 

□  Defense  Acquisition 
Workforce 

□  Defense  Acquisition 
STEM  Workforce 


1946) 


Totals:  6.6M  42.8M  41 .8M  56.7M  7.4M 


NDIA  SE  Conference  Keynote  Address 
10/27/09  Page-12 


UNCLASSIFIED 


The  Timeline  has  Collapsed! 


Conventional  Warfare 


Counter-Insurgency  Warfare 


USAF  Capability  Adversary  Capability 


US  Capability 


Adversary  Capability 


High  Altitude  Aircraft 


Jammers 


Mine  Resistant  Ambush 
Protected  (MRAP) 


Advanced 

Technology 


Response  loop 

Response  loop 

measured  in 

measured  in 

years 

months  or  weeks 
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An  Effective  Process  for  Major 
efense  Systems  -  but  not  very  agile 


Integrated  Defense  Acquisition,  Technology,  and  Logistics  Life  Cycle  Management  System 
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Scope  of  DDR&E  Acquisition 
Program  Oversight  Efforts* 


Program 

Category 

Increasing 

cost/risk 

#  of 
Progs 

AC  AT  ID** 

$$$ 

MDA  =  AT&L 

93 

ACAT  1C** 

$$$ 

MDA  =  CAE 

52 

Special 

Interest** 

Any  $s 
Risk 

19 

MAIS,  ACAT 
IA 

$-$$$,  AIS 

30 

Pre-MDAP 

$$$ 

pre-MS  B 

53 

Pre-MAIS 

$-$$$,  AIS 
pre-MS  B 

10 

ACAT  II 

$$  <  ACAT  1 

8 

ACAT  III 

$  <  ACAT  II 

9 

Total 

274 

*Based  on  2009  T&E  Oversight  List  (Jan  5,  2009) 
**Major  Defense  Acquisition  Program  (MDAP) 
+Major  Automated  Information  System  (MAIS) 


%  Distribution  of  MDAPs 
by  Domain 


□  Land 

■  C2-ISR 

□  Unmanned 

□  Ships 

■  Munitions 

□  Rotary  Wing 

■  Comms 

□  Space 

■  Business 

■  Missiles 

□  Fixed  Wing 

□  Other 


MDA-  Milestone  Decision  Authority 
TMA  -  Technology  Maturity  Assessment 
CAE  -  Component  Acquisition  Executive 
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Defense  Acquisition  Approach 

Systems  Engineering  is  a  key  discipline 


h* 


Typically  5  to  15  Years 


H 


I 


Concept 

Material 

Technology 

Development 

Engineering  & 

Production 
&  Deployment 

Operations 

Engineering 

Solution 

Manufacturing 

MDD 

Analysis 

MS 

MS 

Development 

MS 

i 

•  Intel  review 

•  DP/Early 

A 

•  Threat  assess 

B 

•CDR 

C 

LRIP  i  FRP 

1 

•  Sustainment 

•  Op  needs 

4- 

Systems  Engr 

A 

•  System  Spec 

▲ 

•TRR 

: 

•  Disposal 

•  Tech 

•  AoA 

•SEP 

•  Dev  test  &  eval 

•  Transition 

objectives 

•  Red  Taming 

•  T&E  plan 

•  Initial  Ops  Test 

•  Ops  Test  &  Eval 

•SCR 

•SEP 

•  SRR  &  PDR 

&  Eval 

•  Training 

•  T&E  Strategy 

•  Prototype  dev 

•  PRR 

•TRA 

•TRA  Update 

Technology  and  Risk  Reduction 

•  Technology  “push”  investment 

•  Technology  maturation 

•  Phenomenology  measurements 


70-75%  of  Cost  Decisions  Made  Prior  to 
Milestone  A 

Impact  72%  of  Total  Life  Cycle  Costs 


AoA  -  Assessment  of  Alternatives 
DP  -  Developmental  Planning 
MDD  -  Material  Development  Decision 
SCR  -  System  Concept  Review 
SRR  -  System  Requirements  Review 
SEP  -  System  Engineering  Plan 
PDR  -  Preliminary  Design  Review 
CDR  -  Critical  Design  Review 
TRR  -  Test  Readiness  Review 
PRR  -  Production  Readiness  Review 
LRIP  -  Low-Rate  Initial  Production 
FRP  -  Full  Rate  Production 


NDIA  SE  Conference  Keynote  Address 
10/27/09  Page-16 


UNCLASSIFIED 


DDR&E  Rapid  Fielding  Office: 
Accelerating  Delivery  of  Capabilities 


Problem 

Identification 


JUONS  (65) 
IPLs  (96) 
STIPL  (90) 
COCOM  Requests 
Anticipated  Needs 
Other 


Solution 

Matching 


Implementation 

Tools 


US  Army  REF 
JIEDDO 


US  AF  RCO 
Others 


JCTD 

RRTO 

SCO 

DAC 

FCT 

DARPA 

Other 

Transition 


Program  of 
Record 


Prototype 

Capability 


Lessons  Learned  Feedback 
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Pushing  the  Bounds  of 
Innovation  and  Development 


NlonthS 


CD 

.£ 

£ 

C 
<D 

I 

O 

& 

Q 

c 

<D 

s 

a 

§ 

0 

Q  pec3 


pays 


Ye<*fs 


des 


Rapid  Capability 
Projects 


Tomorrow’s 
Rapid  Capability 
Projects 


+ 


Traditional  platform  acquisition 
programs,  (e.g.  F-22) 


70%  Solution 


More  “Complete”  Solution 


Performance  -  Sustainability  -  Adaptability  -  Robustness  of  Solution 
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A  New  Generation  of 
Concept  Engineering  Tools 


Warfighter  Needs 


Anticipatory 

Opportunities 


/Accelerated  Concept  Engineering  Environment^ 


Iterative  Virtual  and  Real  Prototyping  y 


Conceptual  Designs 

CONOPs 

TTPs 

Detailed  Design  Models 


•  Immersive  Virtual  Environments 

•  Rapid  Virtual  Environment  generation 

•  Virtual  Environment  to  CAD  tool  translation 

•  Rapid  Prototyping  fabrication  tools 

•  “Human-Centered  Design”  principles  and  tools 

•  Integrated  engineering  and  virtual  M&S 
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Summary:  Challenges  Ahead 


*  Develop  tools  to  shorten  the  Acquisition  cycle  without 
diminishing  the  quality  of  solutions 

*  Evolve  Systems  Engineering  to  design  systems  for 
adaptability  and  to  embrace  complexity 

*  Expand  the  aperture  of  Defense  Engineering  to 
address  21st  century  technical  challenges 

*  Expand  the  Defense  Engineering  human  capital 
resource  base 
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Program  Executive  Office 
Command,  Control,  Communications, 

Computers  and  Intelligence  (PEO  C4I) 

Systems  Engineering  Rigor  within 
the  Acquisition  Process 

27  October  2009 
Chris  Miller 
PEO  C4I 
858  537-8779 
chris.miller@navy.mil 


Statement  A:  Approved  for  public  release;  distribution  is  unlimited  (26  OCT  2009) 


T 


Information  Dominance 
Anytime,  Anywhere... 


PEOC4l.NAVY.MIL 


^feXEcur^ 


About  PEO  C4I 


Workforce 
•Civilian:  204 

•  Military:  68 

FY09  Total  Obligation  Authority  (based  on  PB10) 

•  Research  &  Development:  $542M 

•  Procurement,  Navy:  $1,004M 

•  Operations  &  Maintenance,  Navy:  $437M 

•  Ship  Conversion,  Navy:  $1351M 


Platforms  Supported  -  FY09 

•  Afloat:  260  Shore:  220  Expeditionary:  34 

includes:  IAC  -  3  1AM  -  2  (1-DISA/1-PEO  C4I) 
1C -2  PreMAIS/MDAP  - 1 


Navy  C4I  Key  Facts 

More  than  170,000  C4I  users 

More  than  5,200  radios  fielded 

More  than  2,700  annual  installations 

More  than  700  applications  supported 

Average/fielded  bandwidth  capability 
Carrier:  4  mbps  -  24mbps 
Destroyer:  512  kbps  -  8mbps 
Submarine:  128  kbps 

Average  technology  refresh 
18  months 

Average  time  to  market 
Initial  fielding:  36  months 
Full  Fielding:  8-10  years 


Programs  -  Total:  132 

•  ACAT  I:  8*  ACAT  II:  4  ACAT  III  &  Below:  119 

•  Rapid  Deployment  Capabilities  (RDCs):  1 


updated  22  October  2009 
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Unique  Maritime  Challenges  Require 
New  Focus  on  Systems  Engineering 


•  Expansive  Physical  Environment 

>  From  the  ocean  floor  to  outer  space  and 
everything  in  between 

•  High  Volume  of  Data 

>  Linking  Vessel,  People,  Cargo,  Infrastructure 
data  from  multiple  and  disparate  sources, 

>  Then  getting  it  to  the  tactical  edge  in 
a  relevant  format 

•  New  Partners 

>  Traditional:  Coalition  Partners  and 
Interagency  organizations  drive  Cross 
Domain  and  Releasable  Solutions 

>  Non-Traditional:  new  International  and 
Interagency  partners  drive  Non-classified 
solutions 


Information  Technology  is  a  Game-changing  Element  of  Warfare 
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Challenge: 

Realistic  Policy  Implementation 


Law  begets  Policy,  Directives,  and  Guidance 


G3 


Are  we  providing  too  much  “help”? 


Law 


(Titles  10,  40,  44,  ...) 


Development,  complexity, 
and  interpretation  of  Policy 
is  overwhelming 


Is  he  managing  the 
Program,  or  the  paperwork? 
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Need  for  Governance 


Considerably  t'me  and  resources  are 
spent  on  Worthy  and  useful  efforts 
that  are  handicapped  by  a  lack  of  a 
focused,  holistic  integration  concept ' 

_  -  VADM  Dorsett 

^  “  Navy  Integrated  Information  Framework 

-  ’ SkT  \  22  Jun  2009 


Systems  Engineering  Governance 


•  Drivers 

>  Paradigm  shift  in  corporate  culture 

>  Increased  focus  on  fielding  integrated  and 
interoperable  systems 

>  Need  for  up  front  and  early  adoption  of  systems 
engineering  practices 

•  Systems  Engineering  Governance 

>  Technical  Authority  and  Standards 

>  Enterprise  Engineering  and  Certification 


Enabled  Through  a  Competency  Aligned  Organization  (CAO) 
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Competency  Aligned  Organization  (CAO) 


CNO 

Chief  of  Naval  Operations 

— i — 

SPAWAR 

RADM  M.  Bachmann 

VICE  DEPUTY 


ASN(RDA) 

Assistant  Secretary  of  the  Navy 
(Research,  Development  &  Acquisition) 


CURRENT  READINESS 
. REPORTING . 


PEO  C4I 

Mr.  Chris  Miller 


PRINCIPAL  PRINCIPAL 

DEPUTY  MILITARY 

INTELLIGENCE  DEPUTY 


Battlespace  Awareness  & 
Information  Operations 
PMW120 


Deployable  Joint 
Command  and  Control 
PMW140 


International  C4I 
Integration 
PMW  740 


Special  Assistant  for  MDA 
Chief  of  Staff 

DPEO  Acquisition  Management 

DPEO  Manpower  &  Budget 

DPEO  Strat  Mgmnt  and  Process  Improvement 

DPEO  Platform  Integration  &  Modernization 

DPEO  Technical  Direction  &  Program  Integration 


Command  and  Control 
PMW  150 


Carrier  and  Air  Integration 
PMW  750 


Tactical  Networks 
160 


PMW 


Ship  Integration 
PMW  760 


Communications 
PMW  170 


Submarine  Integration 
PMW  770 


Shore  and  Expeditionary 
Integration  PMW 

790 


CAO  implementation  increases  consistency  and 
collaboration  within  engineering  and  acquisition 


Systems  Engineering 
Rigor  Applied  to  Acquisition 


•  Technical  Authority  provides: 

>  Engineering  expertise  during  system  development  and 
deployment 

>  MDA  with  an  independent  assessment  of  program  technical 
health 

>  Consistent  enterprise  standards  and  processes  to  ensure 
interoperability  with  traditional  and  non-traditional  partners 
within  the  GIG 

•  Enterprise  Engineering  and  Certification  (E2C): 

>  Design  system  interoperability  early  in  the  systems 
engineering  lifecycle 

>  Test  end-to-end  capability  packages  for  interoperability 

>  Enforce  acquisition  programs  to  collaborate  on  engineering 
design,  development  and  interoperability  challenges  prior  to 
fleet  installation 
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System  Engineering  Technical  Reviews 
(SETR)  &  the  Acquisition  Lifecycle 


Milestones 

Phases 


A 


A 


A 


Acquisition 
Gate  Reviews 


Material 

Solution 

Analysis 

Material  Development 
Decision 

Q  Q 


Technology 

Development 


Ai 


Engineering  and 
Manufacturing  Development 

Integrated  System  System  Capability  & 

_  Manufacturing  Process 

Design  Demonstration 


is) (o  a 


Production  & 
Deployment 


Operation  & 
Support 


Pass  1 


Pass  2 


Pre-System  Acquisition 


■»  <r- 

TRA  SSR 

V 


System  Acquisition 

IRR 


TRA 

V 


Sustainment 


A 

ITR 

A 

ASR 

1 

A  A 

SRR  1  SRR  II 

1  ,  1 

A 

SFR 

1 

AA  A 

PDR  IBR  PDR 

i  i 

!  or  ! 

A 

CDR 

1 

A 

TRR 

A 

SVR  / 

FCA 

PRR 

A 

OTRR 

A 

PCA 

1 

A 

ISR 

i 

1 

1 

1 

V 

V 

Preferred  System 

System  Specification 
Concept 

Functional  Allocated 

Baseline  Baseline 

Initial 

Product 

Baseline 

Final 

Product 

Baseline 

-A. _ 

-a, _ 

_ 

E2C 


X 

Cross  -  PMW 
Requirement  Spec  Review 


Cross  -  PMW 
Design  Spec  Review 


End  to  End  Test 


Technical  reviews  and  E2C  activities  occur  as  the  system 
matures  throughout  the  program  life  cycle 


NOTE:  Gate  Reviews  are  NOT  SETR  Events 
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PEO  C4I  Masterplan 

Documents  Portfolio  Implementation  across  FYDP  and  beyond 


PEO  C4I  Masterplan 

Version  3.0 


August  7,  2009 

Distribution  D:  Distribution  authorized  to  the  Department  of  Defense  and  U.S.  DoD  contractors  only 
(administrative  or  Operational  Use)  (28  JULY  2009);  other  requests  must  be  referred  to  the  Navy’s 
Program  Executive  Office  for  C4I. 


Purpose-  Provides  an  understanding  of  what 
transition  is  required  across  the  PEO  C4I 
portfolio  in  order  to  meet  modern  network¬ 
centric  warfare  needs 

•what  is  planned  and  budgeted 
•baseline  architectures 
•future  architectures 
•portfolio  roadmaps 
•future  technical  vision 

•recommendations  for  modernization  initiatives 

Intended  Audience-  Intended  to  be  used  as 
a  ready  reference  for  all  PEO  C4I  portfolio 
stakeholders,  including  program  managers, 
resource  sponsors  and  warfighters. 

Updates-  Living  document  updated  annually. 


Available  at: 

https://nserc.navy.mil/peo_c4i/se2/dpeo/dpeotechdir/PEO%20C4l%20Masterplan%20Version%2030/Forms/Allltems.aspx 
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Net-centric  Enterprise  Solutions  for 
Interoperability  (NESI) 


•  “A  distillation  of  several  higher 
level  strategies  into  a 
manageable  set  of  guidance” 

•  Framework  of  actionable 
engineering  guidance 

•  Content  evolves  to  support 
growing  experience  with  net- 
centricity,  interoperability  and 
program  needs 


•  Publicly  available  content 

>  http://nesipublic.spawar.navy.mil 


Master  Plan  utilizes  the  Navy 

Technical  Reference  Model  to  bin  programs  by  functional  area 


Moving  Forward 


•  Increase  focus  on  enterprise  standard 
development  and  implementation 

•  Standardize  processes,  best  practices  and 
lessons  learned 

•  Work  with  stakeholders  to  develop  enterprise 
level  requirements  to  support  the  future 
warfighter 
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We  get  it. 

We  also  integrate  it,  install  it  and 
support  it.  For  today  and  tomorrow. 
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Welc  ome 

to  the 

12th  Annual 

Systems  Engineering  Conference 


Sponsored  by 

NDIA  Systems  Engineering  Division 


In  Conjunction  With 

Director,  Systems  Engineering,  Office  of  the  Deputy  Undersecretary 
of  Defense  (AT&L),  Defense  Research  &  Engineering 

With  Technical  CoSponsorship  By 

Institute  of  Electrical  &  Electronics  Engineers  (IEEE) 
Aerospace  &  Electronic  Systems  Society  and 

Systems  Council 


Systems  Engineering 


Why  do  we  need  systems  engineering? 

*  SE  is  the  all-important  facilitative  umbrella 
engineering  and  “glue”  that  enables  and  fosters 
successful  programs 

*  As  the  US  military  moves  from  a  threat-based 
weapons  procurement  environment  to  a  capability- 
based  procurement  environment,  Systems 
Engineering  becomes  even  more  important. 


Capability-based  Acquisition 


The  US  was  in  a  Threat-based  mode  for  defense 
planning  for  40  years,  from  the  arrival  of  Robert 
McNamara  as  SecDef  in  1961  until  Donald  Rumsfeld 
took  over  in  2001 

Analysts  believe  this  led  to  two  weaknesses: 

-  Near  impossible  to  do  flexible  and  adaptive 
planning 

-  Military  planners  became  so  narrowly  focused 
that  they  missed  potential  dangers 

Gulf  War  was  a  surprise 
Kosovo  was  a  surprise 
Sept  11,  2001  was  a  surprise 
Iraqi  insurgency  was  a  surprise 
Taliban  revival  was  a  surprise 


Capability-based  Acquisition 


In  a  recent  RAND  report,  analyst  Paul  Davis 
observed  that  whatever  its  faults,  capability  based 
planning  has  the  virtue  of  encouraging  prudent 
worrying  about  potential  needs  that  go  well  beyond 
currently  obvious  threats 

Capability-based  Planning,  and  Acquisition,  needs 
competent  systems  engineering — and  this  is  what 
this  Conference  and  the  NDIA  Systems  Engineering 
Division  -  is  all  about. 

— And  unfortunately,  our  DoD  Systems  Engineering 
Capability  has  atrophied  to  a  large  degree 


Dual-track  Acquisition 

System 

To  further  exacerbate  the  problem,  the  recent  (summer 
2009)  DSB  Report  on  “Fulfillment  of  Operational 
Needs”,  chaired  by  former  AT&L  Jacques  Gansler, 
has  recommended  a  dual-track  acquisition  system: 
the  traditional  force  structure  path  and  a  quick- 
response  path  to  rapidly  answer  demands  for  new 
kinds  of  equipment. 

According  to  the  cover  letter,  the  Pentagon  “lacks  the 
ability  to  rapidly  field  new  capabilities  for  the 
warfighter  in  a  systematic  and  effective  way.” 

Such  a  rapid-response  system  will  have  need  for 
competent  systems  engineering 


A  few  thoughts — 

Do  we  need  a  6th-generation  fighter? 

-  Gen  1 :  Jets  -  F-80,  ME-262 

-  Gen  2:  Swept-wings,  range-only  radar  -  F-86,  MiG-15 

-  Gen  3:  Supersonic  speed,  pulse  radar,  target  acquisition 
beyond  visual  range  -  F-105,  F-4,  MiG-17,  MiG-21 

-  Gen  4:  Pulse-doppler  radar,  high  maneuverability,  look-down, 
shoot-down  missiles  -  F-15,  F-16,  Mirage  2000,  MiG-29 

-  Gen  4+:  High  agility,  sensor  fusion,  reduced  signatures  -  Su- 
30,  F-16+,  F/A-18,  Typhoon,  Rafale 

-  Gen  4++:  Electronically  active  scanned  array  radar,  “active” 
waveform-canceling  stealth,  some  supercruise  -  Su-35,  F-15SE 

-  Gen  5:  All-aspect  stealth,  internal  weapons,  extreme  agility, 
full-sensor  fusion,  integrated  avionics  -  F-22,  F-35 

-  Potential  Gen  6:  Extreme  stealth,  efficient  in  all  flight  regimes 
(subsonic  to  multi-Mach),  smart  skins,  highly  networked, 
extremely  sensitive  sensors,  optional  manning,  directed  energy 


A  few  statistics — 


Some  USAF  analyses  claim  that  the  USAF  could  be 
as  many  as  971  aircraft  short  of  its  minimum 
required  inventory  of  2,250  fighters  by  2030. 

Who  built  our  military  airplanes?  By  WWII: 

-  7,890  by  Chance-Vought 

-  8,810  by  Martin 

-  13,575  by  Bell 

-  15,603  by  Republic 

-  17,428  by  Grumman 

-  18,381  by  Boeing 

-  26,154  by  Curtiss 

-  30,696  by  Douglas 

-  30,903  by  Consolidated  Vultee 

-  41,188  by  North  American 


A  few  more  statistics — 


Ops  Tempo  is  high! 

-  USAF  flew  18,422  sorties  in  Iraq  in  2008 

-  Approximately  9,000  projected  for  2009 

-  USAF  flew  about  19,000  close-air-support  sorties  in 
Afghanistan  in  2008 

-  Number  will  double  in  2009 

-  Sustained  high  Ops  Tempo  wears  out  aircraft! 

Despite  recent  program  cuts,  we  will  be  designing 
and  fielding  new  equipment,  and  competent  systems 
engineering  will  be  needed  for  this. 


And  a  few  historical  items — 


October  2009  is  the  50th  anniversary  of  the  first  US 
InterContinental  Ballistic  Missiles 

-  3  long-range,  liquid-fueled  ATLAS  D  missiles,  armed  with 
nuclear  warheads,  went  on  full  combat  alert  at  Vandenberg 
AFB,  California,  on  October  31, 1959 

-  Development  took  14  years 

The  first  Combat  Drones  -  Unmanned  Air  Vehicles  - 
were  deployed  over  North  Vietnam  in  August  1964 

-  Drones  were  manufactured  by  Ryan  Aircraft  in  San  Diego 

-  They  were  dropped  from  C-130  aircraft 

-  Performed  surface-to-air  missile  recon  over  N.  Vietnam 

-  Drones  recovered  near  Da  Nang  Air  base 

•  Engines  commanded  to  shut  down 

•  Parachute  deployed 

•  Helicopter  snagged  the  chutes  and  drones  mid-air 


Program  -  Tuesday  Oct  27 

0815-1200  PLENARY  SESSION _ Bayview  Ballroom 

0840  -  0930:  Keynote  Address 

Hon  Zachary  J.  Lemnios,  Director,  Defense  Research  &  Engineering 
0930  - 1000  COFFEE  BREAK  Regatta  Pavilion  (“Tent”) 

1000  - 1200:  Acquisition  Executives  Panel: 

Mr  Terry  Jaggers,  OUSD(AT&L)DDR&E/SE,  Principal  Deputy 

Moderator 

Mr  David  Ahern,  Director,  Portfolio  Systems  Acquisition, 
OUSD(AT&L) 

Mr.  Thomas  E.  Mullins,  Deputy  Asst  Secretary  for  Plans, 
Programs  &  Resources,  OUSD(AT&L) 

Mr.  Christopher  A.  Miller,  PEO  for  Command,  Control, 

Communications,  Computers  &  Intelligence,  US  Navy 

Mr.  Randall  G.Walden,  Director,  Information  Dominance 
Programs,  SAF-AQ 


Program  -  Tuesday  Oct  27 


1 330-1 51 5  PLENARY  SESSION  Bayview  Ballroom 

Test  &  Evaluation  Executives  Panel:  View  From  The  Top:  How  Can 
Systems  Engineering  Support  Test  &  Evaluation 

1515-1530  COFFEE  BREAK  Regatta  Pavilion(“Tent”) 

1 530-1 71 5  PLENARY  SESSION  Bayview  Ballroom 

SE  and  Acquisition  Reform:  The  Way  Ahead 

1730-1900  Reception  in  Displays  Area  Regatta  Pavillion 


Luncheon  Activities 


Lunches  in  Regatta  Pavilion 
Tuesday 

Mr.  Stephen  Welby,  Director,  Systems  Engineering, 
OUSD(AT&L)DDR&E 

Wednesday 

-  Presentation  of  NDIA  Lt  Gen  Thomas  Ferguson 
Awards  for  Excellence  in  Systems  Engineering 

Individual  (Leadership  &  Practitioner)  &  Group 

-  Presentation  of  DoD  Top  5  Programs  Awards 

Thursday 

Networking  Lunch 


Program  -  Wednesday  Oct  28 


0800-0945 

1  Systems  Engineering  Effectiveness 

Bayview  III 

2  Early  Systems  Engineering 

Bayview  II 

3  Technology  Maturity 

Bayview  1 

4  Test  &  Evaluation 

Mission  1 

5  Human  Systems  Integration 

Mission  II 

6  System  of  Systems 

Mission  III 

7  Program  Management 

Palm  1 

8  Net-Centric  Operations/Interoperability 

Palm  II 

0945  - 1015  COFFEE  BREAK 

Regatta  Pav. 

1015-1200 

All  above  continue,  except  Track  5  is  System  Safety 

-ESOH 

1200-1330 

Awards  Luncheon  in  Displays  Area 

Regatta  Pav 

See  Program  Brochure  for  Wednesday  PM  and  Thursday  sessions 

Some  Logistics  Info— 


Displays  &  Coffee  Breaks  are  in  Displays  area  in  Regatta 
Pavilion.  17  Exhibitors  are  there  to  discuss  their 
capability  in  Systems  Engineering 

Lunches  are  in  Regatta  Pavilion 


And  Special  Thanks  To- 

Technical  Program  Chairs: 

Steve  Henry,  Northrop  Grumman 
Dr.  Tom  Christian,  USAF  AFMC/ASC 

DoD  Partners: 

Stephen  Welby,  Terry  Jaggers,  Jim  Thompson,  Kristen 
Baldwin,  Nic  Torelli,  Chris  DiPetto,  and  Dona  Lee 

Session  &  Track  Chairs: 

WAY  too  many  to  list,  visit  with  them  in  the  sessions- 
NDIA  Meeting  Executive: 

Suzanne  Havelis 

NDIA  Director 

Sam  Campagna 

Promotional  Partners 

Parametric  Technologies  Corp  (PTC) 

University  of  Phoenix 
Lean  Solutions  Institute 


13th  Annual  Systems 
Engineering  Conference 


October  25-28,  2010 
Hyatt  Regency  Mission  Bay 
San  Diego  California 

Call  for  Papers  &  Call  for  Displays 
is  in  your  registration  information 

Papers  Due  Date:  May  30,  2010 


Headquarters  U.S.  Air  Force 


Integrity  -  Service  -  Excellence 

How  Can  Systems  Engineering 
Support  Program  Execution? 

Mr.  Randall  Walden 

Director,  Information  Dominance 
Assistant  Secretary  (Acquisition) 

12th  Annual  NDIA 
Systems  Engineering  Conference 

26-29  Oct  2009 


U.S.  AIR  FORCE 


U.S.AIR  FORCE 


Defense  Acquisition  System 

Sys  Eng  Throughout  Life  Cycle 


&  Manulaoburlng  □aveliipmani  Ph 


Joint  LI 
Capabilities  [ 
Integration  S, 
Development 
System 


LOtftStiCSf 


AOTR 


Defense 

Acquisition 

System 


OTRR 


Systems 

ngineeriru) 


Planning, 


Programming, 
Budgeting 
&  Execution 
Process 


Integrated  Defense  Acquisition,  Technology,  and  Logistics  Life  Cycle  Management  System 


DoD  Acquisition  Chalienges 


U.S.AIR  FORCE 


■  Requirements  Instability 


■  Technology  Maturity 


■  Systems  Engineering 


Integrity  -  Service  -  Excellence 
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Ample  Direction  for  Early  SE 


w 


U.S.AIR  FORCE 


Weapon  Systems  Acquisition 
Reform  Act  (WSARA)  2009 

“Support  each  MDAP  prior  to 
Milestone  A  approval  through  a 
rigorous  systems  analysis  and 
systems  engineering  process” 


k  A 


W  \ 


NRC  Report  “Pre-Milestone  A 
and  Early-Phase  SE”  (Jan  08) 

“Attention  to  a  few  critical  SE 
processes  particularly  during 
preparation  for  MS  A  and  B  is 
essential  to  ensuring  programs 
deliver  on  time  and  on  budget.” 


GAO  Report  -  09-326SP 
“Defense  Acquisitions” 

“ensure  new  programs  follow  a 
knowledge-based  approach  and 
must  begin  with  strong  systems 
engineering  analysis” 


Air  Force 

Acquisition  Improvement  Plan 

“There  will  be  acquisition 
involvement  earlier  in  requirements 
development  process  and  SE 
techniques  will  be  applied” 


Integrity  -  Service  -  Excellence 
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Air  Force  Acquisition 
Improvement  Plan 


U.S.AIR  FORCE 


SECAF  &  CSAF  approved  plan  to  recapture  acquisition  excellence 

Five  Major  Goals 

1.  Revitalize  the  Air  Force  Acquisition  Workforce 

2.  Improve  Requirements  Generation  Process 

3.  Instill  Budget  and  Financial  Discipline 

4.  Improve  Air  Force  Major  Systems  Source  Selections 

5.  Establish  Clear  Lines  of  Authority  and  Accountability 


Acquisition  Improvements  Heavily  Dependent  Upon  SE 


Integrity  -  Service  -  Excellence 
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Focus  on  Air  Force  SE  Processes 


Streamline  Acquisition  Processes 

■  Move  Faster,  Smartly 

Instill  Systems  Engineering  Discipline 

■  Technical  reviews  and  processes 

Active  SE  Early  in  Program  Life  Cycle 

■  Shape  scope,  requirements  definition, 
identify  viable  concepts 

■  Extensive  user/provider  collaboration 


Integrity  -  Service  -  Excellence 
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U.S.AIR  FORCE 


Early  Systems  Engineering  Was  Lost 


Problem  Statement 

“Overstated,  unstable  requirements  that  are  difficult  to  evaluate  during  source  selections’ 


Early  Systems  Engineering 
and  Development  Planning 
virtually  eliminated 


Eroded  acquisition  expertise  in 
translating  ops  requirements  to 
system  requirements 


Capability 

Assessment 


Capability 

Need 

Identified 


Operational 

Requirements 


System 

Requirements 


Eroded  acquisition  expertise  and  processes 
that  supported  the  lead  command  early  in  the 
development  of  program  requirements 


System  Delivery  and  Fielded  Capability 


Solution 

“Ensure  acquisition  involvement  and  leadership  in  support  of  the  lead  command 

early  in  the  development  of  program  requirements” 


Integrity  -  Service  -  Excellence 
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Disciplined  SE  Reduces  Risk 


U.S.AIR  FORCE 


■  Up  to  75%  of  life  cycle  cost  is  determined  during  concept 
refinement  and  requirements  generation 

■  Identify  the  key  decision  points 

■  What  are  the  significant  cost  drivers  -  budgetary  risk 

■  New  technologies  -  engineering  risk 

■  What  does  the  80%  solution  look  like 

■  What  is  commercially  available 

■  What  do  we  prototype 

■  Identify  risk 

■  Cost,  technical,  integration,  manufacturing,  and  sustainment 


Upfront  Effort  And  Resources  Will  Pay  Significant  Life-Cycle  Benefits 


Integrity  -  Service  -  Excellence 
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Air  Force  Vision 


U.S.AIR  FORCE 


■  Attack  problem  early  with  Disciplined,  Repeatable  Processes  from 

JCIDS  CBA  (pre-ICD)  to  AoA,  Pre-MDD 

■  Inform  decision  makers  on  technical  feasibility  of  prospective 
concepts  for  materiel  solutions 

■  Initial  integrated  risk  assessment  addressing  both  operational 
and  programmatic  issues 

■  Support  realistic  program  formulation  through  application  of  early 
Systems  Engineering 

■  Robust  and  disciplined  up-front  technical  planning 

■  Solid  technical  foundation  for  the  future  program 

■  Reduce  the  chances  of  poorly  planned  concepts  emerging  from 
AoA  with  relatively  high  rankings 

■  Use  Concept  Characterization  &  Technical  Description  approach 


Clear  and  Actionable  Policy  and  Process 


Integrity  -  Service  -  Excellence 
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Improving  Program  Execution 


U.S.AIR  FORCE 


■  Development  Planning 

■  Early  and  often  discussions  with  users  to  debate  what  is  feasible 

■  Ensure  ICD/CDD  are  comprehensive,  complete,  and  unambiguous 

■  Analysis  of  Alternatives 

■  Examine  new  applications  of  existing  technologies 

■  Analyze  technical  feasibility  and  risk  of  alternatives 

■  Cost  and  Schedule  Estimates/Execution 

■  SE  is  responsible  for  WBS  development  -  Basis  for  sound  estimates 

■  Independent  assessment  of  contractor  schedule  &  technical  progress 

■  Contracting 

■  Translating  JROC  validated  requirements  into  technical  basis  of  RFP 

■  SE  analysis  key  to  negotiations  with  contractors  on  their  proposals 


Early  Sys  Engineering  Is  Critical  To  Long-Term  Program  Success 


Integrity  -  Service  -  Excellence 
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S E/STEM  Workforce  Initiatives 


U.S.AIR  FORCE 


■  AF  is  establishing  a  Science,  Technology,  Engineering  and  Mathematics  (STEM) 
Emerging  Issues  Panel  to  address  tech  workforce  requirements — to  include  SE 

■  STEM  resources  will  be  affordable,  scalable,  agile,  and  seamlessly  aligned  with  the 
AF  mission  and  strategies 

■  Growing  acquisition  workforce  -  SE  in  high  demand 

■  Additional  hires  for  Product  Centers,  ALCs,  labs  and  other  facilities 

■  Recruiting  additional  Systems  Engineers  using  expedited  hiring 

■  Building  on  and  establishing  aggressive  outreach  efforts  Air  Force-wide  with  our 
high  schools,  colleges,  universities,  sister  services  and  others  existing  efforts 

■  Aggressively  using  the  education  and  training  capabilities  to  keep  AF  STEM 
professionals  current  in  their  fields  and  on  the  cutting  edge  of  technology  - 
we  must  grow  our  technical  expertise  in-house 

■  ‘Bright  Horizons’  strategic  plan  in  development  to  properly  size, 
train,  and  equip  our  SEs/STEM  community  with  the  technical 
depth  and  breadth  needed  for  acquisition  excellence 


Integrity  -  Service  -  Excellence 


U.S.AIR  FORCE 


How  Can  Systems  Engineering 
Support  Program  Execution  ? 


■  Disciplined  Engineering  is  critical  to  program  execution 

■  Early  SE  and  Development  Planning  are  up-front  investments  to 
reduce  risk  in  later  phases  of  the  acquisition  life  cycle 

■  Systems  Thinking  &  Tech  Planning  MUST  start  in  the  early  stages 
of  concept  development,  BEFORE  formal  program  initiation 


■  SE  Experience  is  critical  - 
Invest  and  grow  workforce 


Integrity  -  Service  -  Excellence 
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U.S.AIR  FORCE 


Air  Force  Acquisitions 


Integrity  -  Service  -  Excellence 
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PLENARY  SESSION  2 

Acquisition  Executives  Panel 

View  from  the  Top: 

How  Can  SE  Support  Program  Execution? 

i  Moderator:  Mr.  Terry  Jaggers 

Principal  Deputy,  Systems  Engineering,  Office  of  the  Director, 
Defense  Research  and  Engineering 

]  Mr.  David  G.  Ahern 

Director,  Portfolio  Systems  Acquisition,  Office  of  the  Under 
Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 

i  Mr.  Thomas  E.  Mullins 

Deputy  Assistant  Secretary  for  Plans,  Programs,  and 
Resources  (SAAL-ZR),  Office  of  the  Assistant  Secretary  of  the 
Army  (Acquisition,  Logistics  and  Technology) 

i  Mr.  Christopher  A.  Miller 

PEO  for  Command,  Control,  Communications,  Computers  and 
Intelligence  (C4I),  U.S.  Navy 

i  Mr.  Randall  G.  Walden 

Director,  Information  Dominance  Programs,  Office  of  the 
Assistant  Secretary  of  the  Air  Force  (Acquisition) 


PLENARY  SESSION  4 
Systems  Engineering  Executive  Panel: 

SE  and  Acquisition  Reform: 

The  Way  Ahead 

i  Moderator:  Mrs.  Kristen  Baldwin 

Systems  Engineering  Directorate,  Office  of  the  Director,  Defense 
Research  and  Engineering 

i  Mr.  Nicholas  Torelli 

Systems  Engineering  Directorate,  Office  of  the  Director,  Defense 
Research  and  Engineering 

i  Mr.  Carl  Siel 

Office  of  the  Assistant  Secretary  of  the  Navy  for  Research, 
Development  and  Acquisition  (ASN(RDA)CHSENG) 

i  Mr.  Ross  Guckert 

Office  of  the  Secretary  of  the  Army  for  Acquisition,  Logistics  and 
Technology  (ASA(ALT)) 

i  Colonel  Shawn  Shanley,  USAF 

Chief  Systems  Engineer,  Office  of  the  Assistant  Secretary  of  the  Air 
Force  for  Acquisition,  Science,  Technology  &  Engineering  (SAF/AQR) 


12th  Annual  NDIA  Systems 
Engineering  Conference 

KEYNOTE 


Honorable  Zachary  J.  Lemnios 

Director,  Defense  Research  and  Engineering, 
Office  of  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics) 


Systems  Engineering 

in 

Army  Acquisition 


Mr.  Thomas  E.  Mullins 

Deputy  Assistant  Secretary  of  the  Army 
(Plans,  Programs  and  Resources) 
Office  of  the  Assistant  Secretary  of  the  Army 
(Acquisition,  Logistics  and  Technology) 
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Army  Systems  Engineering  Policy 


The  effective  performance  of  systems  engineering  best  practices  on  a  development 
program  yields  quantifiable  improvements  in  program  execution 
(e.g.,  improved  cost  performance,  schedule  performance,  technical  performance). 


The  Army  System 
Engineering  program 
and  policy  approved 
(13  June  2005  ) 
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•  Requires  a  SEP  for  each  program 

*  Establishes  a  System  Engineer  within 
each  program  and  PEO 


*  Establishes  Army  System  Engineering 
Forum  (ASEF) 
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*  Establishes  peer  review  at  all  major 
technical  reviews 


•  Establishes  the  PEO  as  the  SEP 
approval  authority 
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System  Challenge  for  the  Army 


Army  systems  are  becoming  more  interdependent,  and  required 
operational  capability  is  not  provided  by  a  single  system  but  rather  by  a 
combination  of  systems 

System  of  Systems  (SoS)  Systems  Engineering  (SE)  capability  at  the 
Enterprise  level  is  necessary  to  address: 

-  Stove-pipe  product  development 

-  Many  interdependencies 

-  Path  from  Current  to  Future  Systems 

-  LandWarNet  and  Battle  Command  operations... 

Requirements  are  done  outside  of  the  systems  engineering  process;  we  are  engaging 
the  Training  and  Doctrine  Command  (TRADOC)  on  Command,  Control,  Communications, 
Computers,  Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR)  and  battle 
command  migration,  and  identifying  new  processes  for  system  of  systems  development. 


System  of  Systems  (SoS)  -  A  set  or  arrangement  of  systems  that  results  when  independent  and  useful  systems  are  integrated 
into  a  larger  system  that  delivers  unique  capabilities 

System  of  Systems  Systems  Engineering  (SoS  SE)  -  Planning,  analyzing,  organizing,  and  integrating  the  capabilities  of  a  mix  of 
existing  and  new  systems  into  a  SoS  capability  greater  than  the  sum  of  the  capabilities  of  the  constituent  parts 
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SoS  SE  Management  Structure 


TODAY 


f - \ 

AAE 


|  \ 

MILDEP 


Solutions 
Fielded  But 
Not  Integrated 
Across  Brigade 
Formations 


FUTURE 


•  Leverages  Investment  in  BCT  Modernization 

•Organized  to  Integrate  and  Incrementally  Deliver  Materiel  Solutions  Across  Brigade  Formations 
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SoS  SE  Organization 


MISSION 

Provide  Systems  Engineering  capability  at 
Svstem  of  Systems  level  across  the  Army  enterprise  to 
deliver  integrated  and  interoperable  weapon  systems  that 
provide  optimized  and  affordable  capability 


c  ^ 

Director 

Exec 

(SES) 

Assist. 

Deputy 

Director 


PEOs/JPEOs 
Chief  Engineers 

X  - 


Tasking 

Authority 


Supervisor 


vZ 


Engineer 


FUNCTIONS 

Develop,  evolve,  and  maintain  a  detailed,  interoperable  SoS  design  baseline  -  Enterprise  Systems  Architecture 

Address  technical,  operational  and  cost  aspects  to  frame  issues  for  decision  making 

Leverage  experimentation  and  M&S  tools  as  part  of  engineering  analysis/ operational  assessment 

Establish  and  evolve  an  SoS  vision  over  time,  and  translate  into  capability  attributes 

Translate  emerging  requirements  into  implied  system  attributes  for  technology  insertion  solutions 

Lead  targeted  technical  assessments  to  enable  cost/cap  ability  trades  within  and  across  system  boundaries 

Maintain  visibility'  into  individual  system  architectures,  specifications  &  performance 

Coordinate  technically  with  SEs  in  related  programs  (Army  Joint) 


Engineer 


Engineer 


Engineer 


Supervisor 


Supervisor 


Engineer 


Engineer 


Engineer 


Engineer 


Engineer 


Engineer 


Engineer 
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SoS  SE  Interface 


Design,  develop  and  facilitate  the  delivery  of  relevant,  integrated  and  affordable 
capability  sets  by  type  of  formation  over  time  in  support  of  the  Army’s  modernization 
strategy,  LANDWARNET  and  Army  Force  Generation  (ARFORGEN)  fielding  cycles. 


Army  Acquisition  Executive 


-  Product  Base:  Rqnrits 

-  CONORS  by  Formation 


POM  by  Capabcrly  Pkg 


Disciplined  Process 

(Task  Organized!! 


-  DesgntaCost 

-  Aligi  Rqrnls.  SST,  PQRs,  non-FORs 

-  OpEraliona  Desgn  Revews 

Unit  (Bde)  Architectures 


-  Synchronize  USF  -  Support 
Master  ARFORGEN  schedu  e 

-  Deliver  lime-phased 
capab  ty  sets  overtime 

-RESET 


Leadership  Tasks 
Functional  Tasks 
Specific  T  asks 


Capability  Portfolio  Managers: 

-  ISR,  Gem  ms,  MobfSly,  etc. 

-  Respcnsbie  far  System  Engineer  ng 
Procucte 

-  Cuts  across  PECs  S  Units 


PEOs 


S&T 

RDECs 

DARPA 

UARCs 

Etc. 


r  —  '  “““  . 

Build  Combat  Power  by  Type  Formation  Over  Time  to  Cost 

>>. _ * 
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ggg^ - ASA  {AL&T}  . 

LWN/BC  Capability  Set  Management  Process 


[“Capability  Set  Life-Cycle”  | 


DEFINE  &  DEVELOP 


ickk 


3TE3  1 


CAEILITY  NEEDS  ANALYSIS 


Does  Capability  Set  stand  up  to  Oper  Analysis? 

1  Exercise  Cap  Set  through  Oper  Analysis  - 
overage  analytic  tool  suite 

1  Adjust  to  changes  (funding,  rqimt,  force  changes,  etc.) 
1  Assess  changes  on  SoS  port  &  synchronization 
1  Re-assess  'Bang  for  the  Buck3 


ESTABLISH 

CAPABILITY 

SET 

PARAMETERS 
|QPN.  TECH,  FISCALi 


IR-7  Years 
REFINE 


What  can  Ibe  provided  when  at  affordable  price? 

■  SoS  Engineering  Anaiysis.H'rades 

*  SoS  Synchronization 

*  Technical  Feasibility 

■  Inforrr  decisions 

“Bang  for  the  Buck"  . 


SAP  ABILITY  SEGMENTS 


MZTE-4 


STEP  2 

INTEGRATE 

STEPS  | 

ARCHITECTURES 

F  e r-a  Analysis 

SCREEN  &  ID  SOLUTION  SET 


IT  FOR  CHANGE  ENVIRONMENT 


-ZcMck 

SELECT 

CAPABILITY  SET 
COA 


LWN GOSC 
R-3S  months 


■hV  TECHNOLOGY,  ONSrJUONS,  FORCE  SIZING) 


SOSE  ANALYSIS  OF 
CAPAJ3IUTY  SETS 


Understood  Operational  Effects  Through  Operational  Analysis  (M&S) 


1 1  EC  IP  Lock  |  U 


SYNCH 


Near  Term  Trades 


'Mrtrtr 

APPROVE 

Final 

CAPABILITY  SET 
Syivsn&d  WARFCRGEM 

LWN  GOSC 

R-18  Months 


'Seed  Idea' 
Cut-Off 


|  MTOE  Lockl 


ON&'JUOHS 


S--37l  j, 

SYNCH  * 

CAPABILITY 

SET 

Testing  &  Gertifi  cation 

Forse  Validation  Gcdennoe 
Army  Sourcing  Confere"CM 
Army  Equipp  rig  Conferences 

FIELD 

ARFORGEN  RESET 
SCHEDULE 
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Summary 


System  of  Systems  systems  engineering  is  an  enabler  in  the 
new  Army  BCT  Modernization  strategy. 

Systems  Engineering  is  being  done  in  Army  programs;  we 
need  to  ensure  that  it  is  consistent,  and  consistently  followed 
across  the  PEOs. 

Implement  efforts  to  support  requirements  generation  at  the 
System  of  Systems  or  Enterprise  Level,  and  help  define  the 
trade  space. 
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The  Role  of  Systems  Engineering 

In  Program  Management 

12th  Annual  NDIA  SE  Conference 

Mr  David  Ahern 

Director,  Portfolio  Systems  Acquisition  & 

Functional  Lead  for  Program  Management 

OUSD(AT&L)/A&T(PSA) 
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Unclassified 


Agenda 

•  Major  Policy  Changes — 

•  5000.02 

•  Weapons  Systems  Acquisition  Reform 
Act  (WSARA)  of  2009 

•  Systems  Engineering  Role  in  EVM 

•  Systems  Engineering  In  Reliability 


Unclassified 
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Excerpts  from  '§'&fetary  Gates 
Congressional  Testimony  -  January  27,  2009 


9  The  situation  we  face  today:  A  small  set  of  expensive  weapons 
programs  has  had  repeated  -  and  unacceptable  -  problems  with 
requirements,  schedule,  cost,  and  performance 

9  I  do  not  believe  there  is  a  silver  bullet,  and  I  do  not  think  the 
system  can  be  reformed  in  a  short  period  of  time... 

9  That  said,  I  do  believe  we  can  make  headway,  and  I  have  already 
begun  addressing  these  issues 

9  We  must  freeze  requirements  on  programs  at  contract  award 
and  write  contracts  that  incentivize  proper  behavior 

9  Programs  that  cost  more  than  anticipated  are  built  on  an 
inadequate  initial  foundation.  I  believe  the  Department  should 
seek  increased  competition,  use  of  prototypes,  and  ensure 
technology  maturity  so  that  our  programs  are  ready  for  the 
next  phases  of  development 

Systems  Engineering  plays  a  critical  role  in  changing  the  future 


Unclassified 
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Unclassified 

DoD  Instruction  5000.02  Summary 


•  While  we  have  much  to  do,  the  Department  has  taken 
action  to  address  many  of  the  issues  related  to 
program  execution 

*  Ensuring  a  proper  foundation  before  initiating  programs 

*  Limiting  requirements  changes 

*  Requiring  mature  technologies  and  system  engineering 
discipline 

*  Competitive  prototypes  to  reduce  risk,  improve  competition, 
inform  decisions 

*  Better  integration  between  development  and  operational  test 
and  evaluation 

*  Improvements  in  how  we  incentivize  contract  performance 

*  It  will  take  time  to  realize  the  results  of  these  changes 
...but  we  are  already  seeing  improvements 
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Previous  Acquisition  Process 


Process  entry  at  Milestone  A,  B,  or  C 

Entrance  criteria  met  before  entering 
phase 

Evolutionary  Acquisition  or  Single  Step 
to  Full  Capability 


IOC 


FOC 


Concept 

Technology 

System  Development 

Production  & 

Operations  & 

Refinement 

Development 

&  Demonstration 

Deployment 

Support 

\  Concept 
/  Decision 

/s.  Design 
<  >  Readiness 
V  Review 

LRIP/IOT&E  /S  Decision 

V  Review 

Pre-Systems  Acquisition 


Systems  Acquisition 


Sustainment 


Problems  Identified 

•Most  potential  programs  proceed  to  Milestone  B  without  a  predecessor  review  to  assess  the 
capability  need  and  direct  analysis  of  alternatives 

•Technical  maturity  is  not  adequately  demonstrated  prior  to  program  initiation 

•Program  cost ,  schedule,  and  performance  inadequately  informed  by  design  considerations 

•Requirements  “creep”  continues  to  destabilize  programs 

•With  the  exception  of  Configuration  Steering  Boards  at  the  CAE  level,  there  is  no  formal  and 
effective  opportunity  between  Milestone  B  and  Milestone  C  for  MDA  to  assess  progress,  adjust 
L  defer  requirements,  or ;  consistent  with  statute,  restructure  the  program _ 
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Comparison  to  DoDI  5000.2,  May  12,  2003 


Defense  Acquisition  Management  System ,  May  2003  -  December  2008 


(Program  Initiation) 


B 


Concept 

Refinement 

Technology 

Development 

System  Development  & 
Demonstration 

Production  &  Deployment 

Operations  & 
Support 

(PDR) 

(CD^)  0 

LRIP/IOT&E 

Concept 

Decision 


Pre-Systems  Acquisition  i 


Design  FRP 

Readiness  Decision 

Review  Review 

Systems  Acquisition 


i  Sustainment  i 


Defense  Acquisition  Management  System,  Revised  December  8,  2008 


>/A 


/\  (Program  Initiation) 


S* 


Technology 

Developpjent--^ 

Engineering  &  Manufacturing 
Deyetopttient 

Production  &  Deployment 

/  (PDR) 

\  , . /"> 

\  r^DR)  V 

LRIP/IOT&E  ^ 

Materiel 
Solution 
alysis 


I  III 

|  Post-CDR  J 

FRP 

V  0rl 

_  1 

V  Assessment  / 

Decision 

Operations  & 
Support 


JDR  after’ 

'  w/  Post-PDF 
Assessment 


Review 


Greater  emphasis  upfront — where  systems  engineering  is  most  critical 
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Opportunities  for  SE  to  have  Impact 

— MDD  and  Material  Solution  Analysis 


Technology  Opportunities  &  Resources 


•  Materiel  Development  Decision  precedes  entry  into  any  phase  of 
the  defense  acquisition  management  system 

•  Entrance  criteria  met  before  entering  phase 

•  Evolutionary  Acquisition  or  Single  Step  to  Full  Capability 


MS  C 


Full  Rate 
Production  DR 


Strategic 

Joint 

Guidance 

Concepts 

UU 

OSD/JCS  COCOM 


TechDev 

CDD 

Engineering  & 
Manufacturing 

CPD 

Production  & 
Deployment 

O&S 

J  Development 

o 

Incremental  Development 


> 


•  JROC  recommends  that  the  Milestone  Decision  Authority  (MDA)  consider  potential  materiel  solutions 

•  MDA  ensures  necessary  information  is  available  to  support  the  decision 

•  Materiel  Solution  Analysis  Phase  begins  with  the  MDD — the  formal  entry  point  into  the  acquisition 
process,  mandatory  for  all  programs 

•  At  the  MDD,  the  Joint  Staff  presents  the  JROC  recommendations;  the  DoD  Component  presents  the  ICD 
and  a  preliminary  concept  of  operations,  a  description  of  the  needed  capability  and  operational  risk,  and 
the  basis  for  determining  that  non-materiel  approaches  will  not  sufficiently  mitigate  the  capability  gap 

•  D,PA&E  (or  DoD  Component  equivalent)  proposes  Assessment  of  Alternatives  (AoA)  study  guidance 

•  MDA  approves  the  AoA  study  guidance;  determines  the  acquisition  phase  of  entry;  identifies  the  initial 
review  milestone;  and  designates  the  lead  DoD  Component(s) 

•  Decisions  documented  in  an  Acquisition  Decision  Memorandum  (ADM) 
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Opportunities  for  SE  to  have  Impact 
— Prototyping  and  Competition 


•  The  Technology  Development  Strategy  and  associated 
funding  shall  provide  for  two  or  more  competing  teams 
producing  prototypes  of  the  system  and/or  key  system 
elements  prior  to,  or  through,  Milestone  B.  Prototype 
systems  or  appropriate  component-level  prototyping  shall  be 
employed  to  reduce  technical  risk,  validate  designs  and  cost 
estimates,  evaluate  manufacturing  processes,  and  refine 
requirements.  . . . 


Unclassified 
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Opportunities  for  SE  to  have  Impact 

— Preliminary  Design  Review 


PDR  Before  Milestone  B 


PDR  after  B  and 
Post-PDR  Assessment 


*  Consistent  with: 

*  Technology  Development  Phase 
objectives 

*  Associated  prototyping  activity,  and 

*  The  MDA  approved  TDS 

*  Planning  reflected  in  the  TDS 

*  Establishes  the  allocated  baseline  and 
underlying  architectures 

*  Defines  a  high-confidence  design 

*  Conducted  at  the  system  level 

*  Informs  requirements  trades;  improves  cost 
estimation;  and  identifies  remaining  design, 
integration,  and  manufacturing  risks. 


PDR  has  not  been  conducted  prior  tp  ^ 
n/mesfone  B:  y 

•  Plan  fo^a  PDR  as  soon  as  fusible  after 

program  initiation  S 

•  PDR  report  to  thu  MpA  prior  to  the  Post- 
PDR  Assessment  ^  ^ 

*  Report  reflets  requirements  trades 
basec^uffon  the  PM’s  assessment  of 
copt,  schedule,  and  performance  risk 

•^ofmal  assessment;  results  documerfta<J 
+  in  an  ADM  4 

2009  WSARA  requires  before  Milestone  B  . 


Opportunities  for  SE  to  have  Impact 

— Engineering  and  Manufacturing  Development  Phase 


.  .  .  develop  a  system  or  an  increment  of  capability;  complete  full  system  integration;  develop  an  affordable 
and  executable  manufacturing  process;  ensure  operational  supportability;  implement  human  systems 
integration;  design  for  producibility;  ensure  affordability;  protect  Critical  Program  Information;  and 
demonstrate  system  integration,  interoperability,  safety,  and  utility. 


Integrated  System  Design 

System  Capability  and  Manufacturing  Process  Demonstration 

•  Define  system  and  system-of-systems 
functionality  and  interfaces 

•  Complete  hardware  and  software  detailed 
design  and  reduce  system-level  risk 

•  Establish  product  baseline  for  all 
configuration  items 

Demonstrate  the  ability  to  operate  in  a  useful  way  consistent 
with  the  approved  key  performance  parameters  and  that 
system  production  can  be  supported  by  demonstrated 
manufacturing  processes 

Unclassified 
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I  /nr/pQQ/f/prf 

Opportunities  for  SE  to  have  Impact 


— Post  CDR  Assessment 


•  Considers  whether  the  program  provides  capability  consistent 
with  the  Acquisition  Program  Baseline  (APB)  approved  at 
Milestone  B 


*  MDA  determines  whether 

(1)  an  adjustment  should  be  made,  or 

(2)  the  program  should  be  permitted  to  proceed  without  change 

*  Results  documented  in  an  ADM 


Unclassified 
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Unclassified 

2009  Weapon  Systems  Acquisition  Reform  Act 

Greater^  Emphasis  placed^  on  Systems  Engineering 

□  Establishes  Directors  of  Developmental  Test  and  Evaluation  and 
Systems  Engineering 

•  Newly  created  roles  reporting  directly  to  USD  AT&L,  through  DDR&E 

•  Responsible  for  issuing  joint  guidance  relating  to  the  integration  of 
developmental  test  and  systems  engineering,  and  managing  the  associated 
workforces 

•  Components  required  to  develop  and  implement  plans  to  ensure  they  have  the 
appropriate  resources  for  developmental  testing  and  systems  engineering,  and 
the  two  Directors  are  required  to  assess  these  plans. 

□A  Joint  Annual  Report  to  Congress  (first  one  due  March  31,  2010) 
shall: 

•  Report  on  the  activities  undertaken  during  the  preceding  year  establishing 
Directors  and  accomplishing  policy  and  guidance,  review  and  oversight 

•  Highlight  activities  relating  to  the  MDAPs  for  the  preceding  year  including: 

S  A  discussion  of  the  extent  to  which  the  MDAPs  are  fulfilling  the  objectives  of  their  SEPs  and 
TEMPs 

S  A  discussion  of  the  waivers  of  and  deviations  from  requirements  in  TEMPs,  SEPs,  and  other 
testing  requirements  that  occurred  for  the  MDAPs,  any  concerns  raised  by  such  waivers  or 
deviations,  and  the  actions  taken/planned  to  address  such  concerns. 

S  An  assessment  of  the  organization  and  capabilities  of  the  DoD  for  SE,  development  planning, 
and  DT&E  with  respect  to  MDAPs 


Unclassified 
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Unclassified 


WSARA  SE  Implications  for  Programs 

•  Systems  engineering  and  developmental  test  and  evaluation 
now  recognized  in  law  as  inherently  necessary  in 
requirements  definition,  development  planning,  and  early 
acquisition 

•  Need  for  Program  Office  formation  and  PM  skill-sets  after 
Materiel  Development  Decision  (MDD)  and  prior  to  Milestone 
(MS)  A 

•  Increased  importance  of  the  Technology  Development 
Strategy  (TDS)  (as  a  surrogate  Acquisition  Strategy)  at  MS  A 

•  Earlier  engagement  with  industry  and  different  contracting 
strategies  for  technology  maturation ,  competitive  prototyping, 
data  rights,  Preliminary  Design  Review  (PDR)  before  MS  B, 
etc. 

•  Explicit  need  for  earlier,  formal  SE  process  application  (e.g., 
data,  configuration,  and  risk  management) 


Unclassified 


Unclassified 


Systems  Engineering  in  Earned  Value  Management 

□Earned  Value  is  a  management  technique  that  relates 
resource  planning  to  schedules  and  to  technical,  cost  and 
schedule  requirements 

9  During  the  planning  phase,  an  integrated  baseline  is  developed  by 
time  phasing  budget  resources  for  defined  work. 

9  These  time-phased  “planned  value”  increments  constitute  a  cost 
and  schedule  measurement  baseline 

□There  are  two  major  objectives  of  an  earned  value 
system 

*  to  encourage  contractors  to  use  effective  internal  cost  and 
schedule  management  control  systems;  and  to 

*  Permit  the  customer  to  be  able  to  rely  on  timely  data  produced  by 
those  systems  for  determining  product-oriented  contract  status 

□Success  of  EVM  is  dependant  on  good  technical  performance 
baseline  measures  and  can  be  more  effective  as  a  program 
management  tool  if  augmented  with  rigorous  systems 
engineering  processes 


Unclassified 
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Unclassified 


Systems  Engineering  in  Reliability 

□DOD/  5000.02  Additional  Technology  Development  Phase 
Requirements:  PMs  for  all  programs  shall  formulate  a 
viable  Reliability,  Availability,  and  Maintainability  (RAM) 
strategy  that  includes  a  reliability  growth  program  as  an 
integral  part  of  design  and  development  RAM  shall  be 
integrated  within  the  Systems  Engineering  processes, 
documented  in  the  program’s  Systems  Engineering  Plan 
(SEP)  and  Life-Cycle  Sustainment  Plan  (LCSP),  and 
assessed  during  technical  reviews,  test  and  evaluation 
(T&E),  and  Program  Support  Reviews  (PSRs). 


Unclassified 


15 


How  Systems  Engineering  Can 
Support  Developmental  Test  & 

Evaluation 


Mr.  Chris  DiPetto 
Acting  Director 

Developmental  Test  &  Evaluation 


NDIA  12th  Annual  Systems  Engineering  Conference 
Test  &  Evaluation  Executives  Panel 


T&E  Value  Proposition 


MSA  MSB  MS  C 


Specification 


T&E 

Artifacts 


T&E  Strategy 


-  TEMP 

T&E  Plans  .  T&E  p|ans 

-  T&E  reports 


-  TEMP 

-  T&E  plans 

-  T&E  reports 


-  TEMP 

-  Updated 
T&E  plans 

-T&E  reports 


Reality  meets  Design 


“Ground  truth”  for  better  decisions 


2/2 


2 


T&E  /  SE  Collaboration 


Subsystem 

Developmer: 


Subsystem 

Qualification 


Subsystem 

Production. 


Component 

Developmej: 


Component 

Qualification 


Component 

Production^ 


•  Defining  better  requirements  -  evaluatable,  meaningful 

•  Acquiring  knowledge  of  system  /  subsystem  /  component 
capabilities  /  limitations 

•  Risk  management  -  identification,  mitigation 


3 


DT&E  Bottom  Line 


*  T&E  value  is  delivering  credible  knowledge  for  better  decisions 

*  DT&E  must  lead  with  evaluation  -  testing  supports 

*  DT&E  must  be  timely,  efficient,  &  effective 

*  T&E  developed  knowledge  of  capabilities  /  limitations  should  be 
used  at  the  component,  subsystem,  and  system  level 


The  right  information, 
to  the  right  decision  maker, 
at  the  right  time, 
for  better  decisions. 


2/2 
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How  Systems  Engineering  (SE) 
can  support 

Developmental  Test  &  Evaluation  (DT&E)? 

How  Test  &  Evaluation  can  support 
Systems  Engineering  (SE)? 

Navy  Panelist  Response 

Ms.  Amy  Markowich,  SES 

Department  of  the  Navy  Deputy  Test  &  Evaluation  Executive 
27  Oct  2009 


Ways  SE  can  support  DT&E  -  #  1 


■  Include  DT&E  in  SE  team  that  manages 
requirements  and  translates  them  to  technical 
specifications. 

-  Disparate  interpretations  of  performance  measures 
for  test  indicate  insufficient  early  coordination  of 
program  requirements. 

-  T&E  provides  results  of  performance  measures  and 
does  not  participate  in  negotiation  of  trade  space. 

-  Written  correctly,  tech  specs  would  ensure  technical 
and  operational  T&E  definitional  needs  are  clearly 
set,  reducing  disputes  on  what  constitutes 
requirements. 


Ways  SE  can  support  DT&E  -  #  2 


■  Establish  a  lessons  learned  and  best  practice 
forum  for  Systems  Engineering  Plans  (SEP)  to 
generate  an  effective  planning  tool  for  DT&E 
(and  SE). 

-  Perception  is  the  development  process  and 
structure  of  the  SEP  is  not  effectively  implemented 
or  enforced,  resulting  in  inconsistent  documentation. 

-  T&E  elements  in  OSD  SEP  Preparation  Guide  may 
not  be  as  useful  as  they  need  to  be. 

•  Develop  improved  SEP  guidelines  for  T&E  aspects  as 
needed. 


Ways  SE  can  support  DT&E  -  #  3 


■  Require  cross  training  and  rotational  assignments 
between  SE  and  DT&E  communities  to  improve 
integration,  cooperation,  and  understanding. 

-  Perception  that  SE  and  DT&E  are  separate  communities  who 
acknowledge  each  other’s  importance  but  lack  effective 
collaborative  processes  and  procedures. 

-  Workforce  limitations  and  culture  of  “doing  more  with  less” 
generates  smaller  teams  resulting  in  less  collaboration 
between  skill  sets. 

-  Cross  training  can  help  attack  issue  from  two  perspectives 

•  Broader  knowledge  set  within  constrained  workforce. 

•  Recognition  of  what  T&E  expertise  adds  to  team  and  when  it  is 
needed. 


Ways  SE  can  support  DT&E  -  #  4 


■  Identify  how  DT&E  and  OT&E  (i.e.  Integrated 
T&E)  can  be  better  used  to  help  SE  community 
assess  Interoperability. 

-  Interoperability  assessment,  especially  at  the 
System  of  Systems  level,  presents  a  major 
challenge. 

-  DoD  is  still  working  to  understand  and  develop 
appropriate  evaluation  processes. 

-  -  Representative  joint  mission  threads/environments 
and  CONOPS  are  key  to  assessing  performance. 


How  can  DT&E  support  SE? 


Fulfill  Verification  in  SE 

-  Sync  TEMP  with  SEP  on  CTP  resolution  for  technical  reviews. 

-  Participate  in  RAM  growth  plan  development. 

-  Early  in  development  stress  components  and  systems  to  anticipated 
operational  limits. 

-  Track  status  of  all  deficiencies  identified  during  test. 

-  Formalize  test  result  reporting  within  the  program  to  quickly  provide 
feedback  across  all  working  levels. 

Bridge  Verification  to  Validation: 

-  DODI  5000.02,  EMD  Phase:  “Developmental  and  operational  test 
activities  shall  be  integrated  and  seamless  throughout  the  phase.  ” 

-  Link  Operational  Test  objectives  with  measures  of  technical 
performance  early  in  requirements  generation. 

-  Relate  DT  results  to  impact  on  COI  resolution. 

•  DODI  5000.02,  “T&E  ...  should  be  reported  in  terms  of  operational  significance 
to  the  user.  ” 


Ways  T&E  can  support  SE 


DT  tests  spec  compliance,  OT  test  operational 
environment 

Use  of  Joint  Mission  Threads  across  life  cycle  provides 
realistic  R&D  and  test  scenarios. 

Getting  OT  involved  early  in  SE  process  helps  identify 
operational  issues  that  can  be  corrected  early  in  the 
system  design.  Full  IT  helps  solve  this. 

-  Issues 

•  Does  the  OTA  really  understand  fleet  need 

•  Is  OTA  staffed  to  develop  test  requirements  that  early  in  program 

Potential  for  sharing  of  models,  simulators,  labs, 
scenarios  across  SE  and  T&E 

-  Cost  savings  and  reduction  of  conflicting  analysis 
environments 


Questions? 


“12th  Annual  Systems 
Engineering  Conference 

T&E  Executives’  Panel  on 
Developmental  Test  and 
Evaluation 


PROMOTING  NATIONAL  SECURITY  SINCE  1919 


12th  ANNUAL 


“ACHIEVING  ACQUISITION  EXCELLENCE  VIA  EFFECTIVE  SYSTEMS  ENGINEERING” 


CO-SPONSORED  BY  NDIA  TEST  &  EVALUATION  DIVISION 


Thank  You  For  Silencing  Your 
Cellular  Phones  During  the 
Conference  Sessions 


Achieving  Acquisition  Excellence 
via  Effective  Systems  Engineering 
Building  October  27,  2009 


“One  of  the  primary  and  critically  important 
areas  of  program  acquisition  and  execution  lies 
in  the  umbrella  discipline  of  Systems 
Engineering,  which  is  the  overall  integrating 
function  in  defense  programs,  from  proper 
requirements  definition  &  flowdown,  effective 
and  affordable  design  that  integrates  reliability, 
availability  and  maintainability  considerations 
into  the  overall  balance  of  design  that 
emphasizes  supportability  and  usage  aspects 
along  with  overall  performance,  cost  and 

schedule. 

Systems  Engineering  principles  embody  strong 
technical  and  risk  management  aspects,  for 
both  the  acquiring  program  office  as  well  as  the 
executing  defense  prime  and  subcontractors. 

Strong  emphasis  on  systems  engineering 
throughout  the  life  cycle  of  the  program,  from 
concept  development  through  sustainment,  is  a 
key  enabler  of  successful  programs.” 


NPIfl 


12th  ANNUAL 

SYSTEMS  ENGINEERING 
CONFERENCE 


Achieving  Acquisition  Excellence 
vias  Effective  Systems  Engineering 
October  27,  2009 


“A  major  conference  focusing  on  improving 
acquisition  and  performance  of  Defense 
programs  and  systems,  including  network 
centric  operations  and  data/information 
interoperability,  systems  engineering  and  all 
aspects  of  system  sustainment. 

The  DoD  seeks  to  improve  the  acquisition 
process  and  overall  program  execution  of 
military  systems,  to  provide  greater,  more 
effective  and  reliable  warfighting  capability,  at 
affordable  cost  and  within  reasonable 

schedules. 

The  annual  Systems  Engineering  Conference 
explores  the  role  of  systems  engineering  in 
defense  programs  from  all  aspects  and 
perspectives,  including  the  pragmatic,  practical 
and  academic  viewpoints,  and  brings  key 
practitioners  together  to  work  on  effective 
solutions  to  achieving  a  successful  warfighting 

force.” 


12th  ANNUAL 

SYSTEMS  ENGINEERING! 
CONFERENCE 


OUR  DISTINGUISHED 
PANELISTS: 


Mr.  Christopher  DiPetto,  Acting  Director, 
Test  and  Evaluation,  ODDRE,  Pentagon 


Ms.  Amy  Markowich,  Deputy, 
Department  of  Navy  T&E  Executive, 
Pentagon 


Col  Dexter  Sapinoso,  Chief  Air  Force 
Test  &  Evaluation,  Policy  and  Programs, 
Pentagon 


Dr.  James  Streilein,  Technical  Director, 
Deputy  Commander,  ATEC 


Awaiting  Y  our 
Questions 


“The  DT&E  function  of  OSD  is 
broken  and  in  serious  need  of 
strengthening.  Current  legislative 
proposals  include  establishing  a 
Director,  Developmental  Test  & 
Evaluation  to  address  this 

shortfall.” 


Defense  Science  Board  Task  Force  on 
Developmental  Test  &  Evaluation 

Findings  <&  Recommendations 

•Lack  of  a  disciplined  engineering  process  during 

systems  development. 

•Numerous  attempts  at  acquisition  reform  had 
reduced  discipline  in  program  formulation  and 

execution. 

•Changing  the  test  process  will  not  remedy 
systemic  deficiencies  in  program  formulation  and 

execution. 

•Inadequate  response  to  shortfalls  identified 
during  developmental  test  and  evaluation  (DT&E) 
in  areas  of  reliability,  availability  and 
maintainability  (RAM). 

•Program  constraints,  such  as  schedule  and 
funding,  often  preclude  incorporating  fixes  into 

identified  shortfalls. 


•Defense  Science  Board  Task  Force  on 
•Developmental  Test  &Evaluation 
•[chartered  by  the  USD(AT&L], 


Department  of  Defense 
T&E  Community 


Secretary  of  Defense 


Under  Secretary  of  Defense 

(Acquisition,  Technology  &  Logistics) 


Director  of 
Defense  Research  & 
Engineering 


Secretary  of  the  Army 

Army  T&E 
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Integrity  -  Service  -  Excellence 

How  Systems  Engineering  Can 
Support  Developmental  Test 

NDIA  Systems  Engineering  Conference 

Colonel  Dexter  Sapinoso 

Chief,  Policy  and  Programs 
Directorate  of  Test  and  Evaluation 
Phone:  703  697-0190  or  DSN  227-0190 
E-mail:  Dexter.Sapinoso@pentagon.af.mil 
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U.S.AIR  FORCE 


Five  Things  The 
T&E  Process  Must  Do 


U.S.AIR  FORCE 


■  Support  early  development  of  requirements 

■  Reduce  technical  risk  (CT  and  DT&E) 

■  Test  efficiently  -  avoid  redundant  effort 

■  Collaboration  between  testers,  developers  &  users 

■  Periodic  “vector  checks”  toward  IOT&E 

■  Verify  capabilities  achieved  (IOT&E) 

■  Final  test  report  briefings  to  Air  Staff  and  OSD 


11/2/2009 
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AF  T&E  Structure 


U.S.AIR  FORCE 


11/2/2009 


Integrity  -  Service  -  Excellence 
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U.S.AIR  FORCE 


Systems  Engineering  at  4  levels 


■  Contractor  (Prime  &  Sub)  Level 

■  Design  Definition  based  on  contract  requirements 

■  Attempts  to  tie  design  features  to  contract  requirements 

■  DT  Organization  Level 

■  Utilize  Systems  Engineering  processes  to  improve  testing 

■  Testing  to  requirement  correlation 

■  Improved  Systems-of-Systems  understanding  for  testing 

■  Program  Office  Level 

■  Operational  Requirements  to  System  Design  Requirements 

■  Provide  framework  for  verification  requirements  of  DT 

■  Balance  program  schedule,  risk,  costs  throughout  lifecycle 

■  SAF/AQ  and  OSD  Levels 

■  Evaluate  program  schedule,  risk,  costs  throughout  lifecycle 
at  SAF/AQ  and  DOD  Levels 


11/2/2009 
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Systems  Engineering 


w 
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w 

U.S.AIR  FORCE 


3  Major  Processes  Aligned  - 

Requirements,  Acquisition  &  T&E 


All  3  AFIs  have  same  outline,  use  same  terminology,  and  show  this  figure: 


IOC 


FOC 


AFI  10-601 

(Requirements) 


AFI  63-101 

(Acquisition) 


AFI  99-103 

(Test  &  Evaluation) 


Technology 

Development 


System  Development 

Production 

Operations 

&  Demonstration 

&  Deployment 

&  Support 

LRIP  FRP 

DabNjJj£0*Scised  Test  and  Evaluation  (AFI  99-103)  f 

More  T&E  Involvement  needed  prior  to  MS  B 
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DT  and  SE  Collaboration 


U.S.AIR  FORCE 


■  SE  can  aid  DT  by  providing 

■  Opportunities  for  early  DT  involvement  beginning  at  MS-A 

■  Assistance  to  AF/A3/5  in  requirements  definition 

■  Early  insight  into  schedule  and  risk  assessments 

■  Early  insight  into  technical  and  test  resource  risks 

■  DT  can  aid  SE  by  providing 

■  Early  input  on  definition  and  testability  of  requirements 

■  Insight  into  test  capabilities  and  limitations 

■  Early  detection  of  system  Deficiencies  and  risks 

■  Unique  Lessons  Learned  from  other  programs  earlier  in 
development  cycle 

■  Collaboration  will  improve  early  DT  planning  and 
increase  understanding  of  system  capabilities  and 
risks 


11/2/2009 
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Challenges 


U.S.AIR  FORCE 


■  Requirements  Definition 

■  Ensure  operational  requirements  are  clear,  testable,  and 
verifiable 

■  Ensure  realistic  and  achievable  technological  goals  (  and 
avoidance  of  unobtainable  conditions) 

■  Achieve  buy  in  from  development/acquisition  community 

■  Future  Systems  Integration  and  the  NR-KPP 

■  Requires  significantly  more  systems  engineering 

■  DT  knowledge  of  systems  integration 

■  Systems-of-Systems  integration  and  testing 

■  Balance  of  DT  and  OT,  and  how  to  integrate  them 

■  Development  of  SE  knowledge  base  in  DT 


11/2/2009 


Integrity  -  Service  -  Excellence 


8 


How  Systems  Engineering  Supports 
Developmental  Test  and  Evaluation 


Dr.  James  Streilein 
Technical  Director/Deputy  Commander  ATEC 

Army  Proven 

Battle  Ready 


10/27/2009 


NDIA  12th  Annual  Systems  Engineering  Conference  27  OCT  2009 


JP 


How  We  Fit 


Army  Test  and 
Evaluation 
Command 

MG  Nadeau 


Army 

Evaluation 

Center 

Mr.  Simmons 


Developmental 
Test  Command 

Mr.  Johnson 


Operational  Test 
Command 

Mr.  Amato 


Test  and  Evaluation  (T&E) 
.  Policy  and  Oversight 


Independent  Reporting  Mandated  by  US  Code,  OMB,  and  OSD 


Army  Proven 

Battle  Ready 


ATEC  Mission 

Plan,  test,  independently 
evaluate,  and  report 
throughout  program 
lifecycles  to  advise  combat 
developers/functional 
proponents,  PEOs,  and 
Senior  Leadership 


10/27/2009 
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Systems  Engineering 
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J$lEC  Mission-Based 

Systems  Engineering 

•  Unlike  the  commercial  arena,  systems  engineering  for  military 
applications  must  be  more  rigorous  to  ensure  effectiveness, 
suitability,  and  personnel  survivability  in  the  harshest  environments. 

•  As  such,  effective  systems  engineering  must  expand  requirements 
analysis  into  the  mission  context. 

•  The  expansion  requires  an  understanding  of  the  engineered 
attributes  (function  and  performance)  of  the  system.  Part  of  that 
understanding  is  learned  through  DT. 

•  Mission-based  approach  can  lead  the  way  to  research,  develop,  test 
and  verify  mission  capabilities. 

-  Goal  is  robust  application  for  SoS,  commercial-off-the-shelf  intensive 
systems,  and  recapitalized  systems. 

Army  Proven 

Battle  Ready 


10/27/2009 
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MBT&E and  SE 


Mission-Based  T&E 


Z 


Authoritative 
Task  Lists 


(2)  Define  Mission  Context 

Determine 
Operational  - 
Mission 

Determine 
^  Factors  &  - 

Conditions 

UNDERSTAND  THE  MISSION 


(3)  Develop  Mission 
Tasks 

Conduct  _ 

Mission 

Analysis 

^  Link  to 
Authoritative 
Tasks 

i 

\ 

/ 

(4)  Develop 
Tai 

Supporting 

sks 

Determine 

Conditional 

Tasks 

Determine  , 
Enabling 
Tasks 

(5)  Identify  Task  Capabilities 


Identify 
Capability  ' 
Requirements 


Associate 
Tasks  with 
Required  . 
Capabilities 


Understand 

the 

Mission 


(6)  Determine 
System-of-System 
Components  (physical 
architecture) 


(7)  Develop  System  Attributes 


Identify  Attribute 
Requirements 


Associate 
Components  with 
Attributes 


(8) 

Associate 

Task 

Capabilities 

with 

Component 

Attributes 


UNDERSTAND  THE  SYSTEM 


Systems  Engineering 


Requirements  Analysis 

•  Missions  and  Environments 

•  Functional  Requirements 

•  Define  Performance  Requirements 


Functional  Analysis 

•  Decompose  to  lower  level  Functions 

•  Allocate  Performance 

•  Define  Functional  Interfaces 


Synthesis 

•  Transform  Architectures 

•  Define  Alternative  System  Concepts 

•  Define  Physical  Interfaces 

•  Define  Alternative  Products  &  Process 


Understand 

the 

System 


ATeC  MBT&E  &  SE  Aligned  Goals* 


•  Execute  SE  and  scope  T&E  efforts  earlier  in  the  acquisition 
cycle  based  on  mission  task  capabilities. 

-  Addresses: 

•  “Insufficient  systems  engineering  applied  early  in  the  program  life  cycle...” 

•  “Requirements  not  always  well-managed,  including  the  effective  translation  from  capabilities 
statements  into  executable  requirements...” 

-  By:  Focusing  on  mission  task  capabilities  as  the  starting  point. 


•  Enable  robust  SE  and  T&E  strategy  development  for  Joint  networked  SoS 
and  a  common  environment  for  collaborative  effort  between  capabilities 
developer,  materiel  developer  and  T&E. 

-  Addresses:  “Collaborative  environments,  including  SE  tools,  are  inadequate  to  effectively 
execute  SE  at  the  joint  capability,  SoS  and  system  levels.” 

-  By:  Using  a  framework  that  links  all  components  of  the  SoS  to  the  mission  capability  and  uses 
a  common  definition  of  terms. 


Army  Proven 

Baffle  Ready 


10/27/2009 


*  From  NDIA  SE  Division  Task  Group  Report  July  2006 
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^EO  SE  Connections  to  T&E 

•  T&E  must  be  an  integrated  aspect  of  the  SE  process  from  virtually  the  beginning 
to  the  end. 

-  Both  DT  and  OT  are  integral  to  the  SE  process. 

-  DT  addressing  the  technical  maturity  and  specification  compliance. 

-  OT  addressing  customer  needs  and  satisfaction. 

•  Testers  working  together  with  requirements  systems  engineers  assure 
requirements  and  specifications  are  unambiguous  and  verifiable. 

•  The  role  of  SE  is  determining  and  translating  operational  needs  to  engineering 
specifications.  MBT&E  checks  both. 


•  T&E  supports  the  system  engineering  process  by  turning  information  into 
knowledge. 

-  The  cost  associated  with  obtaining  information  is  not  trivial.  However,  the  cost  of  ignorance  is  huge. 

-  Some  believe  testing  is  expensive  but  fixing  the  problems  found  late  in  the  program  is  far  more 
expensive. 


Army  Proven 

Battle  Ready 


Successful  design  reviews  answer  questions,  assure  risk  is  appropriate  and 
convince  decision  makers  to  approve  moving  into  the  next  phase. 

-  T&E  results  provide  the  most  compelling  rationale. 


10/27/2009 


7 


Challenges 


•  Integrating  DT/OT 

•  Information  Assurance/Network  security 

•  Software 

•  System  of  Systems  integrations 

•  Live,  Virtual,  Constructive  considerations 

Army  Proven 

Battle  Ready 


10/27/2009 
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PLENARY  SESSION  2 

Acquisition  Executives  Panel 

View  from  the  Top: 

How  Can  SE  Support  Program  Execution? 

i  Moderator:  Mr.  Terry  Jaggers 

Principal  Deputy,  Systems  Engineering,  Office  of  the  Director, 
Defense  Research  and  Engineering 

]  Mr.  David  G.  Ahern 

Director,  Portfolio  Systems  Acquisition,  Office  of  the  Under 
Secretary  of  Defense  (Acquisition,  Technology  and  Logistics) 

i  Mr.  Thomas  E.  Mullins 

Deputy  Assistant  Secretary  for  Plans,  Programs,  and 
Resources  (SAAL-ZR),  Office  of  the  Assistant  Secretary  of  the 
Army  (Acquisition,  Logistics  and  Technology) 

i  Mr.  Christopher  A.  Miller 

PEO  for  Command,  Control,  Communications,  Computers  and 
Intelligence  (C4I),  U.S.  Navy 

i  Mr.  Randall  G.  Walden 

Director,  Information  Dominance  Programs,  Office  of  the 
Assistant  Secretary  of  the  Air  Force  (Acquisition) 


PLENARY  SESSION  4 
Systems  Engineering  Executive  Panel: 

SE  and  Acquisition  Reform: 

The  Way  Ahead 

i  Moderator:  Mrs.  Kristen  Baldwin 

Systems  Engineering  Directorate,  Office  of  the  Director,  Defense 
Research  and  Engineering 

i  Mr.  Nicholas  Torelli 

Systems  Engineering  Directorate,  Office  of  the  Director,  Defense 
Research  and  Engineering 

i  Mr.  Carl  Siel 

Office  of  the  Assistant  Secretary  of  the  Navy  for  Research, 
Development  and  Acquisition  (ASN(RDA)CHSENG) 

i  Mr.  Ross  Guckert 

Office  of  the  Secretary  of  the  Army  for  Acquisition,  Logistics  and 
Technology  (ASA(ALT)) 

i  Colonel  Shawn  Shanley,  USAF 

Chief  Systems  Engineer,  Office  of  the  Assistant  Secretary  of  the  Air 
Force  for  Acquisition,  Science,  Technology  &  Engineering  (SAF/AQR) 


12th  Annual  NDIA  Systems 
Engineering  Conference 

KEYNOTE 


Honorable  Zachary  J.  Lemnios 

Director,  Defense  Research  and  Engineering, 
Office  of  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  and  Logistics) 


UNCLASSIFIED 


ASA  (AL&T) 


NDIA  12th  Annual  Systems  Engineering  Conference 


Organizing  for  the  Future 

Army  SE  Initiatives 


ROSS  R.  GUCKERT 

Assistant  Deputy  for  Acquisition  and  Systems  Integration 
Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics  and  Technology 

Ross.Guckert@us.army.mil 

27  October  2009 
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UNCLASSIFIED 


ASA  (AL&T) 


Agenda 


Developmental  Planning  in  the  Army 
Army’s  Capability  Package  Process 

System  Engineering  Enablers 

-  System-of-Systems  Systems  Engineering  (SoS  SE) 

-  PEO  Integration 

Army  Reliability  Initiatives 


Developmental  Planning 

in  the  Army 


UNCLASSIFIED 


ASA 

Army  Developmental  Planning 


Development  Planning 

► 


Development 


t 

)P  (Early  SE) 

SE 

L< 

;s 

DT&E 

Army  Capstone  Concept 
Army  Expeditionary  Warrior  Experiment 
Army  Evaluation  Task  Force 
C4ISR  On-The-Move 


DP:  Development  Planning 

DT&E:  Developmental  Test  and  Evaluation 

SE:  System  Engineering 

CBA:  Capabilities  Based  Assessment 
CDD:  Capability  Development  Document 
CPD:  Capability  Production  Document 
ICD:  Initial  Capabilities  Document 
MDD:  Materiel  Development  Decision 


UNCLASSIFIED 


ASA  (AL&T) 


Army’s  Capability  Package  Process 


UNCLASSIFIED 


Army  Capability  Set  Management  Process 


“Capability  Set  Life-Cycle” 


DEFINE  &  DEVELOP 


What  can  be  provided  when  at  affordable  price? 

•  SoS  Engineering  Analysis/Trades 

•  SoS  Synchronization 

•  Technical  Feasibility 

•  Inform  decisions 

•  “Bang  for  the  Buck” 


Does  Capability  Set  stand  up  to  Oper  Analysis? 

•  Exercise  Cap  Set  through  Oper  Analysis  - 

leverage  analytic  tool  suite 

•  Adjust  to  changes  (funding,  rqmt,  force  changes,  etc.) 

•  Assess  changes  on  SoS  perf  &  synchronization 

•  Re-assess  “Bang  for  the  Buck” 


PRIORITIZE 
^^BILITY 
EDS 


R-7  Years 
REFINE 


ESTABLISH 

CAPABILITY 

SET 

PARAMETERS 
(OPN,  TECH,  FISCAL) 


APABILITY  SEGMENTS 


STEP  4 


STEP3 


Fiscal  Analysis 


SCREEN  &  ID  SOLUTION  SET 


APPROVE 


STEP5 


IT  FOR  CHANGE  ENVIRONMENT 


SELECT 

CAPABILITY  SET 
COA 


LWN  GOSC 

R-36  months 


EW  TECHNOLOGY,  ONS/JUONS,  FORCE  SIZING) 


PRODUCE/ 

PRIORITIZE 

COAS 


SOSE  ANALYSIS  OF 
CAPABILITY  SETS 


Understood  Operational  Effects  Through  Operational  Analysis  (M&S) 


i&rMt 

Approve 

“Baseline” 

CAP.  SET  15-16  FOR 
REFINEMENT 

LWN  GOSC 

R-6  Years 


Develop 
“BASELINE” 
INTEGRATED 
CAPABILITY 
SET 


SYNCH 


I  i  ill  I  I 

80%  Solution’ 


Near  Term  Trades 


APPROVE 

Final 

CAPABILITY  SET 
Synched  wARFORGEN 

LWN  GOSC 

R-18  Months 


“Good  Idea” 
Cut-Off 


ONS/JUONS 


> 


STEP7 


FIELD 

ARFORGEN  RESET 
SCHEDULE 


SYNCH 


CAPABILITY 

SET 

Testing  &  Certification 


MTOE  Lock 


Force  Validation  Conference 
Army  Sourcing  Conferences 
Army  Equipping  Conferences 


UNCLASSIFIED 


ASA 

Notional  Timeline  for  Capability  Sets 


Month  1 


Month  2 


Month  3 


Month  4 


UNCLASSIFIED 


- ASA 

Task  Force  120  -  Overview  of  CO  As 


FCS  NIKs 


JCR  key 
leaders 


Common  to 
all  COAs: 

CO  CP  comms, 
WIN-T  Incr  2, 
FCS  SO  sensors 


COA  3 


EPLRS 
for  FCS  NIKs 


Rifleman  Radio 


PRC-117G 
for  FCS  NIKs 


GMR 


COA  Id 


50%  GMR 


50%  MP 


COA  1 


MP 


SFF-S 


COA  1b 


COA  1c 


COAle 


UNCLASSIFIED 


CO  Cdr 

(GMR J* 


SRW  Secret 


ASA  (AL&T) - 

Task  Force  120  —  Recommended  Course  of  Action 


BFT 


NCW  (Ka/Ku) 


AN/PRC  119  (SINCGARS  VHF) 


UAV  \  N, 

Feed 


XSRW  Secret: 
^  hityjiited  and 
Dismot^FTeci 
HMS  MPrfc, 


BN  CP  •■4  BN  CP 

b4*"  UiNiR 


BN  CDR 

iflfc 


AN/PRC  148  (VHF  UHF) 

242 

AN/PRC  150  (HF) 

8 

AN/PSC  5 

20 

AN/VRC  104  V3  (HF) 

63 

SINCGARS  VEH  RTs(VHF) 

1313 

AN/PRC-154  (RR) 

1658 

COA  1  b/1  c/1  d/1  e 

JTRS  WNW 
(GMR) 


RF  7800  retained  as 
TPEfor ATH 


SINCGARS 

(individual  voice  nets  not  shown) 


f  Key  Changes 

'  -WINT  Inc  2 
-CO CP  with  WINT  tnc  2TBD  Cl  (not  PoP.) 


(HlflS  fe’-S) . 

SQl5"""^  SQI] 


FCSSensors  use 
SRW  to  reachBkit 
with  GMR  or  MR 


,p-  «■*  SK, 

fe  8^. 


-JTRS  GMR  (limited  quantity  for  IBCTs  in  FY11)  to  give  CPs  "Jump"  OTM 

connectivity  and  interoperability  with  multiple  legacy  systems 

-JTRS  Man  Pack  extends  networking  from  soldier  on  JTRS  PLT  up  to  CO/BN 


CSS  VS  AT 


DVB-RCS  (GBS) 


GRRIP  (HUMINTTMS) 
HCLOS 


Assumes  availability  of  HMS  Manpack  (same  as  COA  1  b) 

-  Comms  within  CO  uses  SRW  (instead  of  WNW) 

-  FCS  NIKs  below  CO  use  HMS  MPs  (-50%) 

Assumes  availability  of  SFF-S 

-  Single  channel  radio,  shares  SINCGARS  vehicular  adapter 

-  Postulated  as  low  cost/SWAP  approach  for  vehicles  that  only  require  one  SRW  channel  as  an  advanced  waveform 

-  For  key  leader  JCR  vehicles 

i  As  alternative  to  HMS  manpack 
i  SRW  net  for  more  robust  comms  (augments  BFT) 

-  SFF-S  is  not  currently  part  of  JTRS  program  of  record 
For  dismounts 

-  Rifleman  Radio  up  to  Platoon  Leader 

-  Retain  PRC-148  down  to  Squad  Leader 


RF  7800W  38 

SMART-T 

TROJAN  SPIRIT  V3 
TSR-8  (GBS) 

WIN-T  Inc  2  TCN 
WINT  Inc  2  TBD  ATH  LOS  and  BLOS  13 

WINT  Inc  2  PoP 
WINT  Inc  2  SNE _  34 

WIN-T  LAW  (WIN-T  INC  2  VWP) _ 10_ 

JTRS  GMR _ 40_ 

JTRS  HMS  MP _ 116 

BFT1  -  Ground _ 282 

JTRS  SFF-S  ~60~ 

SO  B-Kit  ICS  81 

SO  B-Kit  Antenna _ 81 

Urban  -  Unattended  Ground  Sensors  29 

(U-UGS) 

Tactical  -  Unattended  Ground  Sensors  12 

(T-UGS) 

Non-Line  of  Sight  Launch  System 
(NLOS-LS) 

Small  Unmanned  Ground  Vehicle  38 

(SUGV)  Block  1 

Class  I  Block  0 _  22 

Other  Hardware  (Inmarsat  BGAN,  S- 
PoP) _ 

SO  A-Kit  HMMWV  81 


UNCLASSIFIED 


ASA 

Army  Network  Modernization  Strategy 


CS  11-12 


CS  15-16 


FY11  FY12  FY13  FY14  FY15  FY16  FY17  FY18  FY19  FY20 


Network 

ncrenhent 


A 

UBC  Vers  2.01 


NetWork 
increment  2 


“Bridging  Strategy’ 


A 

UBC  Vers  3 


Network 

ncrement 


NetworkSynergy 


LANDWAR^ET/GNEC 


APPLICATIONS 


SERVICES 


Capability  Set  and  POR  specific  ARFORGEN  focused  fieldings  that  introduce  new  technologies 
reallocates  assets,  divest  older  technologies  and  incrementally  modernizes  the  Army's  network. 


TRANSPORT 


System  Engineering  Enablers 

-  System-of-Systems  Systems  Engineering  (SoS  SE) 

-  PEO  Integration 
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UNCLASSIFIED 


ASA 

ASA(ALT)  Management  Structure 
for  Army  Mod  Plan 


TODAY 


AAE 


MILDEP 


PM  FCS 


GCV 

Spin 

Network 

Outs  . 

PEO  A 


PEOB 


PEO  C 


PEO  D 


REF 


Non- 

PORs 


Solutions 
Fielded  But 
Not  Integrated 
Across  BDE 
Formations 


FUTURE 


Performs 
Integration 
Function 
Across  Brigade 
Formations 


PEO  A 


PEOB 


PEO 

Integration 


\ 

c - \ 

( - \ 

PEO  C 

PEO  D 

v  J 

) 

i  v  -T '  ~ 

Mod  PM  I  p*— j 


TF  YBDE 
Mod  PM 


Formerly  PM  FCS 


TF  WBDE 
Mod  PM 


k.' 


'  GCV 

•  Increments 

•  Network 


"A 


INTEGRATION 

Formation 
Based 
Task  Forces 


TF  ZBDE 
Mod  PM 


Leverages  Investment  in  FCS  and  OIF/OEF  Procurements  - 
Organized  to  Integrate  and  Incrementally 
Deliver  Materiel  Solutions  Across  Brigade  Formations 


J12 


UNCLASSIFIED 


SOS  SE  Strategic  Goal  -  Warfighters  have  what  they  need,  when  they  need  it,  and  it  works. 

SOS  SE  Vision  -  The  SOS  SE  organization  leads  the  synchronization  of  Army  technical  efforts  and  enables  delivery  of 

world-class  integrated  materiel  solutions  to  the  Warfighter. 

SOS  SE  Mission  -  Architect  and  enable  the  incremental  delivery  of  relevant,  integrated  and  affordable  capabilities  by  formation  type 
in  support  of  the  Army’s  guidance,  modernization  strategy,  and  Army  Force  Generation  model. 

SOS  SE  Stakeholder  Values 

•  Ensure  materiel  solutions  (including  systems,  components,  applications  and  networks)  work  properly  together 

•  Provide  authoritative,  suitable,  relevant,  responsive,  flexible,  integrated,  interoperable,  synchronized,  balanced  SOS  architectures 

•  Coordinate  and  synchronize  efforts  across  PEOs  and  external  entities  (Materiel  Enterprise  (ME),  ARSTAFF,  ATEC,  OSD) 

•  SoS  SE  policy,  guidance,  enterprise  governance  and  terms  of  reference,  define  system  interfaces  and  implementation  of  technical 


standards 

Agile,  responsive,  synchronized  SoS  SE  in  support  of  the  acquisition  process  to  deliver  capability  in  accordance  with  ARFORGEN 
Establish  a  uniform  set  of  Modeling  &  Simulation  and  analysis  tools 

Synchronize  decomposed  requirements  and  adjudicate  conflicts  and  duplications  with  requirements  community 
Shape  technology  transition  to  ensure  greatest  enterprise  value 

SoS-level  Analysis/Trades  to  provide  objective  recommendations  in  operational  terms  (with  TRADOC)  to  enable  better  Army  and  DOD 
level  decisions 

Adjudication  of  cross  PEO  level  SoS  issues 


SOS  SE  Strategic  Objectives 

•  Synchronize  acquisition  program  requirements  and  programmatics 

•  Use  SOS  SE  efficiencies  to  improve  capabilities  delivered  despite  fiscal  constraints 

•  Be  a  recognized  source  for  authoritative  SOS  acquisition  decision  data 

•  Provide  authoritative  SOS  architectures  for  all  Army  formations 

•  Shape  tools  needed  to  execute  SOS  SE  mission 

•  Establish  systems  engineering  enterprise  standards 

•  Shape  S&T  investment  strategy 


UNCLASSIFIED 


-lASA  (AL&T)  — 

Responsibilities  -  SoS  SE,  PEO  I,  Task  Forces 


SoSSE 


PEO 

Integration 


Task  Forces 


Representative,  authoritative  architectures  for  each  Brigade  type 
COA  development,  analysis,  costing  and  decision  support 
Establish  operational  value 

Establishing  standards  and  policy  (via  MILDEP  policy  or  AAE  ADM) 
Alignment/reconciliation  of  resourcing  and  requirements  with  HQDA  and  TRADOC 
Direct  and  lead  SoS  trades 
Resolve  conflict,  provide  governance 

Synchronize  PEOs  with  Army  Mod  Strategy  and  delivery  of  Capability  Packages  - 
Maintain  strategic  IMS/IMP 


Specific  architectures  for  Brigades  to  be  "touched"  in  ARFORGEN 
COA  "executability"  determination 

Execute  SoS  Trades  ICW  ASA(ALT),  provide  recommendations  to  ASA(ALT) 
Recommend  resource  and  requirement  changes  to  align  PORs/non-PORs  to 
Capability  Packages 

Maintain  IMS/IMP  for  all  Brigades  in  ARFORGEN  cycle 

Development  and  management  of  vehicle-network  architectures,  as  well  as 

other  critical  interface/touch  point  architectures  IAW  established  standards 


Manage  IMS/IMP  for  specific  Brigades  to  be  "touched" 

Coordinate  across  PEOs  to  deliver  IAW  architectures  and  IMS/IMP 
Ensure  policy/standard  implementation 
Manage  SoS  testing/certification 
Synchronize  Unit  Set  Fielding 

Recommend  resourcing  changes  to  accommodate  Capability  Package  fielding 


ASA  (AL&T) 


Army  Reliability  Initiatives 
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UNCLASSIFIED 


ASA  (AL&T) 

Army  Reliability  Policy 


Mandates  development  and  demonstration  of  a  mid-SDD  reliability  test 
threshold  for  all  pre-Milestone  B  programs  with  a  JPD  of  JROC  Interest1: 

-  Default  value  is  70%  of  CDD  reliability  requirement 

-  Must  be  demonstrated  with  at  least  50%  statistical  confidence  by  end  of  the  first  full-up, 
system-level  developmental  test  event  of  SDD 

-  Threshold  value  must  be  approved  as  a  part  of  the  TEMP,  and  recorded  in  the  SDD  contract 
and  APB  at  Milestone  B 

-  Requires  review  of  material  developer’s  reliability  case  documentation 

•  AMSAA  and  AEC  to  apply  Reliability  Scorecard 


ATEC  to  perform  threshold  assessment,  and  lead  I  PR  in  event  of  a  breach: 

-  PEO/PM  develops  corrective  action  plan 

-  AEC  performs  assessment  of  PM’s  plan  and  projected  reliability 

-  AMSAA/AEC  estimates  ownership  cost  impacts 

-  TRADOC  assesses  utility  of  system  given  current  reliability  maturity  level 

-  ATEC  CG  provides  recommendation  to  ASA(ALT)  thru  Army  T&E  Executive,  with  PEO 
coordination  in  advance 


ASA(ALT)  policy  expands  the  Army’s  current  T&E  mission 


1.  Per  CJCSI  31 70.01  F,  JROC  “Interest”  refers  to  programs  that  have  a  potentially  significant  impact  on  joint  warfighting. 


UNCLASSIFIED 


Enabling  Early  SE 

5-Step  Arrnv  Policv  Implementation  Plan 


Technology  Development  Phase 


System  Development  &  Demonstration  Phase 


1. 


Establish 


Establish  test  threshold  value12 


Default  value  is  70%  of  requirement,  and  must  be 
demonstrated  with  at  least  50%  statistical  confidence. 


Document 


CDD6 


RFP1’2’4 


RIWG  sample  RFP  language  (DAU  Website),  GEIA-STD-0009 


SDD  Contract1-2'4 

-N 

TEMP1-2 

APB1 

y 

Threshold  to  be  approved  as  part  of  TEMP  and 
incorporated  in  SDD  contract,  TEMP,  and  APB. 


Plan 


Develop  RG  Planning  Curve12 


Source  Selection  Support1’' 


AMSAA  Reliability  Growth  (RG)  Methodology 


Breach  Contingency  Planning 


1, 2,3,5 


Evaluate 


Evaluate  RG  Plan1-2-4 

Threshold  Assessment2 

Report 


ESR/CIPR2 

SEP2 

RIWG  Reliability  Engineering  Scorecard  (DAU  Website) 


Identify  LCC  Impacts1'3'5! 


Sys  Eng.  Plan12 


Milestone  B  OAR  Risk  Assessment2 


Threshold  Breach  Report2-3  5 


j  Only  done  if  threshold  breached.  I  OTA  Assessment  Report2 


Key  players:  1  PEO/PM,  2  AEC-RAM,  3  AEC-ILS,  4  AMSAA  -  Reliability  Branch,  5  AMSAA  -  Resource  Studies  Branch,  and  6  TRADOC. 
Documentation:  Currently  developing  an  ATEC  guide  on  this  implementation  plan  and  associated  reliability  growth  planning  pjr^cesses. 
Reference:  ASA(ALT)  Memorandum,  Dated  6  December  2007,  Subject:  Reliability  of  U.S.  Army  Materiel  Systems. 

GEIA:  Government  Electronics  and  Information  Technology  Association. 


UNCLASSIFIED 


ASA  (AL&T) 


Summar 


•  Agility  and  Responsiveness  are  Critical 
Attributes  for  Army  Acquisition 

•  The  Army  Must  Organize  for  Success  to 
Execute  the  Army’s  Modernization  Strategy 

•  We  Must  Leverage  Enablers  to  Deliver 
Warfighting  Capability 
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Headquarters  U.S.  Air  Force 

Integrity  -  Service  -  Excellence 

Air  Force  Activities  in  Support  of  WSARA 

NDIA  12th  Annual  Systems  Engineering 

Conference 
27  Oct  2009 


Colonel  Shawn  “Magnum”  Shanley 

Chief,  Engineering  &  Technical  Management  Division 

SAF/AQR 

shawn.shanlev@pentaqon.af.mil 

U.S.  AIR  FORCE 


Integrity  -  Service  -  Excellence 


Agenda 


U.S.AIR  FORCE 


■  Background 

■  AF  Acquisition  Improvement  Plan  &  WSARA 

■  Problem  Statement 

■  Early  SE  &  Earlier  SE  -  Study  Recommendations 

■  Employing  Early  Systems  Engineering 

■  Increased  Focus  on  Systems  Engineering 

■  Addressing  the  Content  Gap  -  Current  State 

■  Concept  Characterization  &  Technical  Description 
(CCTD)  approach 

■  Path  Ahead 
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USAF  Flight  Path 


U.S.AIR  FORCE 


Air  Force  2008-2010 
Strategic  Plan 

■  Reinvigorate  AF  Nuclear 
Enterprise 

■  Win  Today’s  Fight 

■  Develop  &  Care  for 
Airmen  &  Families 

■  Modernizing  Aging  Air  & 
Space  Inventories 

■  Recapture  Acquisition 
Excellence 


Acquisition  Improvement  Plan  (AIP) 

■  Revitalize  the  Air  Force  acquisition 
workforce 

■  Improve  requirements  generation 
process 

■  Instill  budget  and  financial 
discipline 

■  Improve  Air  Force  major  systems 
source  selections 

■  Establish  clear  lines  of  authority 
and  accountability  within 
acquisition  organizations 
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U.S.AIR  FORCE 


AF  Acquisition  Improvement 

Plan  (AiP)  &  WSARA 


Acquisition 
Improvement  Plan 


Revitalize  the  Air 
Force  acquisition 
workforce 


Improve  requirements 
generation  process 

Establish  clear  lines 
of  authority  and 
accountability  within 
acquisition 
organizations 

Instill  budget  and 
financial  discipline- 

Improve  Air  Force 
major  systems 
source  selections 


Weapon  System  Acquisition  Reform  Act  (PL  111-23) 

Sec.  102  Directors  of  Developmental  Test  and  Evaluation 

and  Systems  Engineering 

(a)  In  General 

§  139d.  (b)  (5)  Director  of  Systems  Engineering  shall 

(D)  provide  advocacy,  oversight,  and  guidance  to  elements  of  the  acquisition 

►  workforce  responsible  for  systems  engineering,  development  planning,  and 
lifecycle  management  and  sustainability  functions; 

(E)  provide  input  on  the  inclusion  of  systems  engineering  requirements  in 

►  the  process  for  consideration  of  joint  military  requirements  by  the  Joint 
Requirements  Oversight  Council  ... 

(b)  Developmental  Test  and  Evaluation  and  Systems  Engineering  in  the  Military 
Departments 

(1)  Plans.  --  The(SAE)  ...  shall  develop  and  implement  plans  to  ensure  the 
military  department ...  has  provided  appropriate  resources  for  ... 


(B)  Development  planning  and  systems  engineering  organizations  with 
adequate  numbers  of  trained  personnel  in  order  to 
(i)  support  key  requirements,  acquisition,  and  budget  decisions  made  for 
each  major  defense  acquisition  program  prior  to  Milestone  A  approval  and 
Milestone  B  approval  through  a  rigorous  systems  analysis  and  systems 
engineering  process;  ... 

(iii)  identify  systems  engineering  requirements,  including  reliability, 
availability,  maintainability,  and  lifecycle  management  and  sustainability 
requirements,  during  the  Joint  Capabilities  Integration  Development  System 
process,  and  incorporate  ...into  contract  requirements  ... 
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Background/Problem  Statement 


U.S.AIR  FORCE 


Problem  Statement 

•  “Overstated  and  unstable  requirements  that  are  difficult  to  evaluate  during  source  selections” 

•  “Ensure  acquisition  involvement  and  leadership  in  support  of  the  lead  command  early  in  the 
development  of  program  requirements” 

Reference:  Acquisition  Improvement  Plan,  4  May  09 


Events  -  circa  1989-1993  -  and  their  unintended  consequences 

•  Acquisition  Workforce  reduction 

-  Early  Systems  Engineering  and  Development  Planning  functions  virtually  eliminated 

-  As  a  consequence,  there  was  erosion  of  acquisition  expertise  and  processes  that  supported 
the  lead  command  early  in  the  development  of  program  requirements 

•  Ownership  of  funding  and  program  advocacy  shifted  from  AFSC  to  MAJCOMs 

-  Further  eroded  acquisition  early  involvement 

•  Congress  “zeroed”  Development  Planning  PE 

•  Significant  reduction  of  specifications  and  standards  -  thru  1998 

-  Shifted  burden  to  industry 

-  Acquisition  expertise  in  translating  operational  requirements  into  system  requirements  eroded 
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So  What? 


U.S.AIR  FORCE 
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Get  Back  to  Early  SE  Roots? 


U.S.AIR  FORCE 


■  Weapon  Systems  Acquisition  Reform  Act  2009 

■  “Support... each  MDAP  prior  to  Milestone  A  approval .. .through  a 
rigorous  systems  analysis  and  systems  engineering  process” 

■  Acquisition  Improvement  Plan 

■  “There  will  be  acquisition  involvement  earlier  in  the  AF 
requirements  development  process  and  systems  engineering 
technigues  will  be  applied” 

■  NRC  “Pre-Milestone  A  and  Early-Phase  Systems  Engineering” 
study  committee  report  (Jan  08)  - 

■  “Attention  to  a  few  critical  SE  processes  ...  particularly  during 
preparation  for  Milestones  A  and  B  is  essential  to  ensuring  ... 
programs  deliver  ...  on  time  and  on  budget.” 

■  GAO  Report  -  09-326SP  “Defense  Acquisitions” 

■  “...  ensure  new  programs  ...  follow  a  knowledge-based  approach 
...  must  begin  with  strong  systems  engineering  analysis” 


Integrity  -  Service  -  Excellence 


National  Research  Council 
Study  Recommendations 


U.S.AIR  FORCE 


1.  Air  Force  leadership  should  require  that  Milestones  A  and  B  be 
treated  as  critical  milestones  in  every  acquisition  program  and 
that ...  the  “Pre-Milestone  A/B  Checklist”  ...  be  used  to  judge 
successful  completion. 

2.  Assess  career  field  needs  and  develop  a  program  to  address 

3.  Pre-A  decisions  should  be  supported  by  rigorous  SE  processes 
and  analyses  involving  teams  of  acquirers,  users,  and  industry 

4.  A  development  planning  function  should  be  established  in  the 
military  departments  to  coordinate  the  concept  development 
and  refinement  phase  of  all  acquisition  programs  to  ensure  that 
the  capabilities  ...  as  a  whole  are  considered  and  that  unifying 
strategies  such  as  ...  interoperability  are  addressed. 
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Other  Support  Major  Policy 
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U.S.AIR  FORCE 


Increased  Focus  on 
Systems  Engineering 


/v 


y  SECAF  “Go  Fix  SE”  direction  -  2002 
y  DODI  5000.2  -  Mar  03 
SE  Plan  Requirements  Policy  Memo  -  Mar  07  y 

USD  (AT&L)  RAM  Memo  -  Jul  08  y 
USD  (AT&L)  Lifecycle  Mgt  Framework  Memo  -  Jul  08  ▼ 

SAF/AQR  CCTD  Guidance  Memo  -  Dec  08  y 

DODI  5000.02  -  Dec  08  y 
CJCSI  3170.01  -  Mar  09  y 
Weapon  Systems  Acq  Reform  Act  -  May  09  y 


co 

_i 

~0 

C 

CO 


y  Center  for  Systems  Engineering  Standup  2005 
NRC  Early  SE  Study  -  Jan  08  y 
AF  SE  Assessment  Model  (SEAM)  -  Aug  08  y 

Early  SE  Guide  -  Mar  09  y 
DP  Strategic  Plan,  Continuous  Capability  Planning  Guide  -  Jul  09 

Concept  Maturity/DP  Workshop  -  Aug  09  ▼ 


CO 

0 


o 

< 


V 


> 


2002 


2003 


2004 


2005 


2006 


2007 


2008 


2009 
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U.S.AIR  FORCE 


Current  State 

Filling  the  Gap  Between  CBA  and  MOD 


Technology  Opportunities  &  Resources 


Strategic 

Joint  / 

Guidance 

Concepts\ 

OSD/JCS  COCOM 


MS  C 


Full  Rate 
Production  DR 


Materiel 

Solutions 

Analysis 


Technology 

f=[ 

Engineering  & 

Production  & 

Develop¬ 

CDD 

Manufacturing 

CPD 

Deployment 

O&S 

ment 

Development 

o 

AoA 


Incremental  Development 


> 


Currently  little  if  any  “Space” 
between  ICD  and  MDD  ... 

This  is  where  Early  Systems 
Engineering  has  the  most 
leverage  for  future  program 
success ... 
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U.S.AIR  FORCE 


Early  Systems  Engineering 

Vision 


■  Disciplined,  repeatable  processes  from  JCIDS  CBA 
(pre-ICD)  to  AoA;  Pre-MDD  Focus 

■  Inform  decision  makers  on  technical  feasibility  of  prospective 
concepts  for  materiel  solutions 

■  Initial  integrated  risk  assessment  addressing  both  operational  and 
programmatic  issues 

■  Support  realistic  program  formulation  through 
application  of  early  Systems  Engineering 

■  Robust  and  disciplined  up-front  technical  planning 

■  Solid  technical  foundation  for  the  future  program 

■  Reduce  the  chances  of  poorly  planned  concepts  emerging  from  AoA 
with  relatively  high  rankings 

■  Extensive  user/developer  collaboration  early  &  throughout  lifecycle 

Clear,  Actionable  Policy  &  Process 
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w  Early  Systems  Engineering,  Capability 
Planning,  Development  Planning,  and  5000.02 


U.S.AIR  FORCE 


Capability  Planning 
(DOTMLPF) 


ICD 

DCR 


User  Needs, 

Validated 

Requirements 

'  JCIDS  outputs 
(if  available) 

'  Capability 
shortfall 
•  Others 


Development  Planning 
(prospective  materiel  solutions) 


MSA 


AoA  Study 
Guidance 


JROC 
AFROC 


Trade  Space 
Characterization 


CCTDs 


JROC 
AFROC 


Release 

Approval 


Final 

Concepts 

Review 


Programmatic 

Analyses 


Candidate 
Solution  Sets 
Selection 


Concept 

Characterization 

Review 


Initial 

Concepts 

Review 


Candidate  Solution  Sets 
Characterization 
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U.S.AIR  FORCE 


Concept  Characterization  & 
Technical  Description  Elements 


1.  Mission  /  Capability  Need  Statement  /  CONOPS 

2.  Concept  Overview 

3.  Trade  Space  Definition  /  Characterization 

4.  Studies,  Analyses,  Experiments 

5.  Concept  Characterization  /  Design 

6.  Program  Characterization 

7.  Risk  Assessment 

8.  DOT_LPF  Implications 

9.  Conclusions  (Capability  Description;  Traceability  to  Need  Statement) 


Tailored  to  meet  MDD  needs 
Tailored  to  timeline  needed  to  support  decision 
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CCTD  Summary 


U.S.AIR  FORCE 


■  Supports  AoA  Study  Guidance  and  MDD 

■  Documents  application  of  disciplined  early  Systems 
Engineering  processes 

■  Serves  as  “concept  spec”  or  initial  technical  baseline 

■  “Living  document”  that  later  supports  development  of 

■  Technical  Requirements  Document  (TRD) 

■  System  Requirements  Document  (SRD) 

■  Generally  executed  by  Product  Center  Capability 
Development  /  Planning  /  Integration  offices  (XR) 

■  Assistance  from  decision  makers,  user,  HAF/A5,  AFMC/A5, 
AFMC/EN,  Center/EN,  AFRL,  others 
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Path  Ahead 


U.S.AIR  FORCE 


■  IN  WORK 

■  Institutionalize  CCTD  process  across  Product  Centers 

■  Clarify  CCTD  descriptions;  develop  Guidebook 

■  Simplify  implementation 

■  Provide  template  for  authors  to  follow 

■  Update  Early  SE  Guide  -  set  and  enforce  policy 

■  Address  resource  requirements 

■  FUTURE 

■  Flesh  out  “Collaborative  Development  Centers” 
concept  for  use  across  all  Product  Centers 
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U.S.AIR  FORCE 


Conclusion:  Development 
Planning  Is  A  Team  Sport! 


■  To  be  effective,  development  planning  is  a  collaboration 
among  communities 

■  Systems  engineering  provides  a  structured,  disciplined 
approach  as  a  basis  for  this  collaboration 
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Prevent  This! 
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U.S.AIR  FORCE 


Integrity  -  Service 


Backup 


-  Excellence 
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SE  Is  One  of  Many  Enabling  Processes 


Development  Planning 


U-S-  AIR  FORCE 


L 


CBA 


MDD 


Materiel  Solution  Analysis 


MS  B 


Technology  Development 


Users 


Statement 
of  User 
Needs 


Independent 

Analysis 


System 

Engineering 


Programmatic 

Planning 


Development  Planning 


Begins  before  acquisition 

Natural  application  of  systems  engineering  process 

Ensures  that  alternative  system  approaches  evaluated  during 
MSA  are  validated 
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Where  Does  Development 

Planning  Fit? 


Needs  and 
Opportunities 

IPL 

Urgent 

Need 

New  Threat 

New 

Technology 

New 

Strategic 

Direction 

JCIDS 


Viable  Solutions 


Translating  User 
Needs  and 
Opportunities  Into 
Viable  Solutions 


Delivery  Options 

Rapid  Fielding 
Technology  Refresh 
O&S  Upgrade 
Experimentation 

JCTDs 

5000.02 


MS  A 


MS  B 


MS  C 


c,  .  .  Joint 

S"?*e9'c  Concepts 
Guidance  r 

CBA 

ICD 

1  Materiel 

Solution 
^MDjJ>  Analysis 

Technology 

Development 

* 

CDD 

Engineering  and 
Manufacturing 
Development  and 
^^Demonstration 

CPD 

Production  and 
Deployment 

^ 

O&S  1 

o  * 

PDR  CDR  Full  Rate  Production 

Decision  Review 

Applies  more  broadly  than  JCIDS  to  5000.02  acquisition 
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Topics 
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Chief 
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Engineer 


DoD  5000  and  Weapon  System  Acq  Reform  Act 
DoN  Acquisition  Governance 
Mission  Level  System  Engineering 
System  Engineering  Workforce 


Acquisition  Areas  of  Emphasis 


IRda 

Chief 

Systems 

Engineer 


♦  DODI  5000.02 

-  Earlier  definition  of  requirements  KPP/KSA  feasibility 

-  Technology  Development  Strategy  and  System  Engineering 

-  Use  of  prototyping  during  TD  phase 

-  T&E  Strategy 

-  Total  Ownership  Cost 

-  Cost  Estimates  to  Budget 

-  Sustainment  /  Logistic  Planning  and  Execution 

-  RMA  considerations 


♦  Weapon  System  Acquisition  Reform  Act 

-  Development  and  tracking  of  measurable  performance 
criteria 

-  Competitive  prototyping 

-  Role  of  systems  engineering  in  development  planning, 
lifecycle  management  and  sustainability 

-  Completion  and  MDA  assessment  of  system  level 
Preliminary  Design  Review  before  MS  B 
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DoN  Acquisition  Governance 


Rda 

Chief 

Bystems 
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♦  The  Secretary  of  the  Navy 

-  Comprehensive  review  of  the  Acquisition  process 

-  Challenges  in  Program  Planning  and  Execution. 

♦  Enhance  the  Acquisition  Governance  process 

-  Inject  Early  Senior  Leadership 

-  Continuous  Engagement  and  Transparency 

♦  Increase  discipline  during  each  phase  of  Program  Maturity 

♦  Codified  by  SECNAVNOTE  on  26  February  2008 


“Two  Pass  /  Six  Gate” 
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DoN  Acquisition  Governance 
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Chief 
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Engineer 


PEO/SYSCOM/ 

OPNAWHQMC 

LEVEL 


♦  First  Pass  -  Requirements  Establishment 

♦  Second  Pass  -  Acquisition  Execution 

♦  Gates  -  Reviews  to  Assess  Readiness  to  Proceed 

♦  System  Design  Specification  -  Capability  and  Performance 
Expectations 
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Gate  Review  Process  Updates 
to  Align  with  DoD  5000  and  WSARA 

^ _ A _ A 


JCIDS 

Process 


Materiel 

Solution 

Analysis 


Technology 

Development 


Engineering  &  Manufacturing 
Development 


IOC 

Production  & 
Deployment 


ICD 


AoA 


CDD  SDS 
CONOPS 


R:P 


0  Materiel 

Development 

Decision 


0 


TD  PDM 


Post 

IBR 


Post 

CDR 


CPD 


Pre 

FRP  DR 


/  \  Post  PDR 
\  /  Assessment 


OPost  CDR 
Assessment 


MS  C 
PDM 


Sustainment 


OFRP 

Decision 

Review 


Technical 

Reviews 


AA 

ITR  ASR 


AA  A 

SRR  SFRPDR  0r 


▲  A 

PDR  IBR 


▲ 

CDR 


A  A  A  A 

TRR  SVR  /  OTRR  PCA 
FCA  / 

_ PRR _ 


[Rda 

Chief 

Systems 

Engineer 


FOC 


Operations  & 
Support 


> 


Disposal 


A 

ISR 


Logistics 

Reviews 


A 

ILA 


A 

ILA 


A  A 

ILA  ILA 


A 

ILA 


♦  Total  Ownership  Costs 

•  Affordability  assessments  earlier 

♦  Life  Cycle  Sustainment  Planning  and 
Execution  including  RMA 

♦  Operational  Manpower  Estimates 


♦  Program  and  Technical  Baseline 

•  Earlier  look  at  KPP  /  KSA  feasibility 

•  Emphasis  on  Prototyping,  Tech 
Development,  and  System  Engineering 

♦  Dev,  Integrated,  and  Operational  T&E 
Planning  and  Execution 

•  T&E  Deficiency  resolution 
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Program 

Health 


4  Factors 


18  Metrics 


Criteria* 


Naval  Probability  of  Program  Success  v2 


Rda 

Chief 
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1  oooo 

OOOO 

oooo 

oooo 

*  Criteria  are  Gate-  and  Metric-specific.  The  number  of  Criteria  will  vary. 

NDIA  12th  Annual  SE  Conference 


Net-Centric  Integration  and  Interoperability 


YSTEMS 

Engineer 


Navy  Battle  Group  Operations:  1997  - 1998 

Impact  of  System  Deficiencies: 

•  Disrupted  CINC  Deployment  Plans 

•  Perturbated  Program  Execution 


What’s  Needed?  . . .  Elevating  Systems  Engineering  to  a  New  Level 

. the  Mission  Level 


Budget  and  Timelines 

Caused  Nearly  10%  Program  Growth 


CNO  WASHINGTON  DC  021648Z  MAY  98 

“The  introduction  of 
increasingly  complex  warfighting 
capabilities  into  the  fleet  has 
resulted  in  significant  battle  group 
interoperability  challenges.  ” 


USS  John  F  Kennedy  (CV  67) 
ACDS  Block  1  Level  2.1 
CEC  B/L  2 


USS  Mahan  (DDG  72) 
USS  Barry  (DDG  52) 
AWS  MK  7  B/L  5.3.6.3 


USS  Hue  City  (CG  66) 
USS  Vicksburg  (CG  69) 
AWS  MK  7  B/L  6  Ph  1 
CEC  B/L  2 

JFK  BG 


USS  Mitscher  (DDG  57)  USS  Cape  St.  George  (CG  71) 
USS  Arleigh  Burke  (DDG  51)  USS  Anzio  (CG  68) 

AWS  MK  7  B/L  5.0.Z5/5.3.5  AWS  MK  7  B/L  5.C.5 

CEC  B/L  1 

IKE  BG 


4INCLANTFLT  BGSIT  021731ZMAR98 

BGSIT  Hot  Wash-Up  Message 

This  report  highlights  the  complexity  of  BG  system 
Architecture,  lack  of  systems  successful  integration  and 
failure  of  critical  equipment. 

In  combination,  the  factors  created  an  incoherent 

tactical  picture  for  BG  operators.” 
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Engineering  at  the  Mission  Level 


Rda 

Chief 

Bystems 
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PLATFORM 


SYSTEM 


COMPONENTS 
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Engineering  Practices  Eda 

at  the  Mission  Level  Systems 

_  Engineer 


Net-Centric  Integration  &  Interoperability 


ISP  Review 

NRKPP 

Guidebook 


DODAF  Usability  For  Engineers 

-Architecture  Data  Elements 
-Architecture  Repository 
-Architecture  Hierarchy 


Mission/SoS 

Engineering 

Guidebook 

Mission  Area 
CHENGs 


Mission 

Operations 

Requirements 


/ 


Validation 


o 


i 


Leveraging  Modeling  and  Simulation 


Large  Scale  Capability 
Assessments 

COTF/MCOTEA  -  Test  in 
the  intended  Environment 


Meta  Data  Strategy 

Analysis  -  Engineering  -  T&E  Community 
Mission  Threads  Mission  Based  Test  Scripts 


Use  of  Live  Virtual  Constructive  Environment 
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The  Engineering  Workforce 


[Rda 

Chief 

Systems 

Engineer 


All  Engineering  Occupations  (08xx) 

•  39,474 


Systems  Planning,  Research. 
Development  &  Engineering  (SPRDE) 

•  17,649 


14,142 

Engineering 


Engineering 
Civilian  Community" 


3,210 


Civilian  Community  \ 

l  22,651 

V  (SPRDE)  I 

297 


Comp  Sci, 
Scientist& 
Physicist 


Enviro,  Indus, 
Civil  Eng 


DoN  Engineering 
Civilian  Community 

•  36,793 


2,384 

Enviro,  Indus,  Civil 
Eng 


Figuring  Out  Who  We  Are  Managing 

OPM,  DAWIA  and  Other  Grouping  Constructs 


Engineering  Community 
Workforce  Geographic  Location 


Rda 

Chief 

Bystems 

Engineer 


7r244 

3r985 

725 

73 

1 


Alabama 

17 

Indiana 

1627 

Nebraska 

2 

Rhode  Island 

1613 

Arizona 

58 

Iowa 

0 

Nevada 

33 

South  Carolina 

977 

Arkansas 

1 

Kansas 

0 

New  Hampshire 

4 

South  Dakota 

0 

California 

7243 

Kentuckzy 

0 

New  Jersey 

669 

Tennessee 

28 

Colorado 

0 

Louisiana 

40 

New  Mexico 

18 

Texas 

81 

Connecticut 

137 

Maine 

1309 

New  York 

19 

Utah 

12 

Delaware 

0 

Maryland 

4630 

North  Carolina 

621 

Vermont 

0 

District  of  Columbia 

2082 

Massachusetts 

85 

North  Dakota 

1 

Virginia 

6001 

Florida 

1832 

Michigan 

1 

Ohio 

3 

Washington 

2755 

Georgia 

201 

Minnesota 

0 

Oklahoma 

11 

West  Virginia 

9 

Idaho 

31 

Mississippi 

305 

Oregon 

2 

Wisconsin 

2 

Illinois 

55 

Missouri 

14 

Pennsylvania 

1228 

Wyoming 

0 

Data  Current  as  of  March  2009 

CONUS  Total 

33,757 

OCONUS  Total 

1,682 

Grand  Total 

35,439 
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Don  syscom 

SEDP 


Workforce  Development  Continuum 


[Rda 
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Bystems 

NGINEER 


In-Service  Workforce 


-  Job  Assignments 

-  Work  Experience 

-  DAWIA  Certification 

-  Graduate  Education 

-  Other  Training 


Undergraduate  Workforce 

-  COOP/Summer  Jobs 

-  Internships 

-  Scholarships 

-  Collaborative  Research 

-  Recruiting 


Future  Workforce  (K-12) 

-  Tutoring 

-  Competitions  (Robotics,  ROVs,  etc) 

-  Science  Fairs 

-  Influencing  Educators  /  Curriculum 
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CHIEF  /  LEAD  SYSTEMS  ENGINEER 

Training,  Qualification  &  Certification  Program  systems 

ENGINEER 


DoN  se  training 


POD  TRAINING 


> 


DoN 

EDUCATION 


PERFORMANCE 


SPRDE 
SE/PSE 
LEVEL  l/ll/lll 


•  Naval  SE  Guide 

•  SEP  Development 

•  Technical  Authority 

•  ECPs  for  Engineers 

•  Leadership 

•  Communication 

•  SYSCOM  Unique 
Courses 


DoN  TWH  CERTIFIES 


SYSCOM 
CHIEF  /LEAD 
SYSTEMS 
ENGINEER 


•  DEMONSTRATED  ABILITY  TO  PLAN  and  IMPLEMENT  ENGINEERING  PROCESSES 

•  BREADTH  AND  DEPTH  OF  TECHNICAL  EXPERIENCE 

•  8-10  YEARS  OF  EXPERIENCE 
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Naval  Systems 


© HIEF 

Bystems 

Engineer 


SHIPS  AND  AIRCRAFTCARRIERS 


SUBMARINES 


AIRCRAFT 


C4ISR  SYSTEMS 


WEAPON  SYSTEMS  LAND  COMBAT  SYSTEMS 
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Unclassified 


“Engineering”:  Overlapping  Taxonomies 


Each  of  the  “Engineering”  workforces  is  defined  by  a  separate  taxonomy: 

o  “0800-Engineering  &  Architecture”  is  1  of  23  white  collar  occupational  groups 

■  Source:  OPM’s  Handbook  of  Occupational  Groups  and  Families 

■  Scope:  Covers  Federal  civilian  workforce 

Occupational  Group  -  A  major  category  of  white  collar  occupations,  embracing  a  group  of  associated 
or  related  occupations;  e.g.,  the  Engineering  and  Architecture  Group,  GS  -0800.... 

O  “Engineering  Civilian  Community”  is  1  of  22  Navy  Civilian  Communities 

■  Source:  Civilian  Community  Management’s  Community  Definition  (also  adopted  by  CHR) 

■  Scope:  Covers  Navy  civilian  workforce 

Community  -  A  subset  of  the  organization's  workforce,  grouped  from  the  highest  organizational 
perspective  by  similarity  of  occupation,  competencies,  and  career  experience.  The  purpose  of 
communities  is  to  cultivate  and  manage  a  set  of  skills  in  the  workforce,  across  the  programs,  lines  of 
business,  departments,  or  lower  level  organizational  units. 

O  “Systems  Planning,  Research,  Development,  &  Engineering”  is  3  of  15  DoD/DoN  AT&L 
Position  Categories 

■  Source:  DoD/DoN  DAWIA  Operating  Guides 

■  Scope:  Covers  DoD/DoN  active,  reserve,  and  civilian  acquisition  workforce 

Position  Category  -  subsets  of  AT&L  positions  that  are  characterized  by  a  common  set  of  core 
acquisition  and  functional  competencies. 
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Unclassified 


Taxonomies :  Authority/Policy 


o  “0800-Engineering  &  Architecture”  - 

part  of  OPM’s  Occupational  Definition 


■  US  Code,  Title  5 


O  “Engineering  Civilian  Community”  - 

part  of  Navy  Civilian  Community  Management  (N1 1 1  Ys  Community  Definition 


■  CEB  Decision  Memo  (Jul  01 ) 

■  OPNAVNOTE  5430 

■  TFPM  MOA,  dated  May  14,  2008 

O  “Systems  Planning,  Research,  Development,  &  Engineering” - 

part  of  DoD/DoN’s  AT&L  Position  Categories 

■  DAWIA  -  Defense  Acquisition  Workforce  Act  (US  Code,  Title  10,  Chapter  87) 

■  DOD  DAWIA  Operating  Guide 

■  DoN  DAWIA  Operating  Guide 


Related  Efforts: 

■  OSD/Component  Functional  Community  Managers  (CFCM) 

•  DOD1 1400.25 

■  SECNAVINST on  Civilian  Competencies  and  Community  Management 

•  Draft  in  routing 


Unclassified 


20  August  2009 
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Weapon  Systems  Acquisition  Reform  Act 

(WSARA)  2009 

OSD  Systems  Engineering  Perspective 

Mr.  Nicholas  Torelli 
October  27,  2009 
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WSARA:  Two  Positions  Established: 
Directors  of  DT&E  and  Systems  Engineering 


Adds  new  section  to  US  Code  Title  10,  Chapter  4:  Sec.  139d.  Director  of 
Developmental  Test  and  Evaluation;  Director  of  Systems  Engineering:  joint  guidance 


•  Director  of  Developmental  Test  and  Evaluation  (DT&E)  AND 
Director  of  Systems  Engineering  appointed  by  the 
Secretary  of  Defense;  report  to  USD(AT&L) 

•  Principal  advisors  to  SecDef  and  USD(AT&L)  on 
developmental  test  and  evaluation  AND  on  systems 
engineering  and  development  planning,  respectively  in  DoD 

•  Two  Directors  will  closely  coordinate  to  ensure  that  the 
developmental  test  and  evaluation  activities  of  DoD  are  fully 
integrated  into  and  consistent  with  the  systems 
engineering  and  development  planning  processes  of  the 
Department 
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Systems  Engineering  Duties: 
Establish  Policy  and  Guidance 


•  Use  of  systems  engineering  principles  and  best  practices, 
generally; 

•  Use  of  systems  engineering  approaches  to  enhance 
reliability,  availability,  and  maintainability  (RAM)  on  major 
defense  acquisition  programs  (MDAPs); 

•  Development  of  SEPs  for  MDAPs  including  systems 
engineering  considerations  in  support  of  lifecycle 
management  and  sustainability;  and 

•  Inclusion  of  provisions  relating  to  systems  engineering  and 
reliability  growth  in  requests  for  proposals; 
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Systems  Engineering  Duties: 
Review  and  Oversight 


•  Review  and  approve  the  SEP  for  each  MDAP; 

•  Monitor  and  review  the  systems  engineering  and  development  planning 
activities  of  the  MDAPs; 

•  Provide  advocacy,  oversight,  and  guidance  to  elements  of  the 
acquisition  workforce  responsible  for  systems  engineering,  development 
planning,  and  lifecycle  management  and  sustainability  functions; 

•  Provide  input  on  the  inclusion  of  systems  engineering  requirements  in 
the  process  for  consideration  of  joint  military  requirements  by  the  Joint 
Requirements  Oversight  Council,  including  specific  input  relating  to 
each  capabilities  development  document; 

•  Periodically  review  the  organizations  and  capabilities  of  the  military 

departments  with  respect  to  systems  engineering,  development 
planning,  and  lifecycle  management  and  sustainability,  and  identify 
needed  changes  or  improvements  to  such  organizations  and  capabilities 
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WSARA:  Roadmap  for  2010  Joint 

Report 
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DoD  Developmental  Test  and  Evaluation  and 

Systems  Engineering 
Joint  Annual  Report  Outline 


1.0 

2.0 

3.0 

4.0 

5.0 

6.0 

7.0 

8.0 

9.0 

10.0 


Executive  Summary 
Introduction 

Overview  of  Military  Department  Assessments 

Assessments  of  Military  Departments’  Developmental  Test  &  Evaluation 

Assessments  of  Military  Departments’  Systems  Engineering 

OSD/DT&E  Activities  &  Oversight  Functions 

OSD/SE  Activities  and  Oversight  Functions 

Assessments  of  MDAPs 

SECDEF  Comments 

Acronyms 


SE-WSARA  Brief  for  NDIA  10/27/09 
10/06/09  Page-1 


UNCLASSIFIED 


Proposed  MDAPs  for 
Joint  Report  -  SE 


Air  Force 

-  BCS-F  Inc  3  (Battle  Control  System-Fixed  Increment  3) 

-  C-1 30  AMP  (C-1 30  Aircraft  Avionics  Modernization  Program) 

-  CITS  (Combat  Information  T ransport  System) 

-  HC/MC-130  (HC/MC-130  Recapitalization  Program) 

-  ISPAN  Bk  1  (Integrated  Strategic  Planning  and  Analysis  Network-Big; 

-  MPS  l-lll  (Mission  Planning  Systems  Increments  l-lll) 

-  SDB  II  (Small  Diameter  Bomb  Increment  II) 

Army 

-  ACS  (Aerial  Common  Sensor) 

-  FCS  (Future  Combat  Systems) 

-  IAMD  (Army  Integrated  Air  Missile  Defense) 

Navy 

-  CH-53K  (Heavy  Lift  Replacement  Progra 

-  H-1  UPGRADES  (4BW/4BN)  (Unite  State 
and  UH-1N  Utility  Helicopter) 

-  JHSV  (Joint  High  Speed  Vessel) 

-  SM-6  (Standard  Missile-6) 

-  SSC  (Ship-to-Shore  Connector) 

-  SSN  774  (VIRGINIA  Class  Submarine) 

Joint 


ea^Marinap 


n«3orps  Mi 


Blogfl) 

'N 


ps  Mid-lifeUpgrade  to  AH-1 W  Attack  Helicopter 


-  JCA  (Joint  Cargo  Aircraft)-Army  Executive  Agent 

-  JTRS  GMR  (Joint  Tactical  Radio  System  Ground  Mobile  Radio) 

-  MIDS  (Multi-Functional  Information  Distribution  System  (Includes  Low  Volume  Terminal  and  Joint 
Tactical  Radio  System)) 
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WSARA  Report 
Near-Term  Schedule 


Near  Term 

S June  26  /  June  30  /  July  8  :  OSD  SE/DT&E  Planning  Meetings 
S July  15:  first  Working  Group  meeting  with  Components 
S July  21 :  SE  Forum  -  Briefed  SE  Component  Leaders  on  plan 

S Aug  10:  Memo  staffed  through  DDR&E  to  USD  (AT&L) 

S Aug  13:  USD(AT&L)  Memo  to  Components  to  support  Joint  Report  Congressional  timeline 
S Aug  24:  D/SE  Memo  to  Components  to  support  SE  portion  of  Congressional  Report  timeline 
S Aug  26:  D,MA  meeting  to  establish  WSARA  Policy  and  Guidance  WG 
^Sept  3:  D,SE  update  meeting 

^Sept  15:  WSARA  OSD  Working  Group  (OWG)  meeting 
^Sept  18:  Kickoff  with  Components  on  actual  SE  reporting  requirements 
^Sept:  Initial  DP/SE/DT&E  meeting  for  Section  102d  Reporting  Requirements 
S October  1:  OSD/SE  work  on  OSD  portion  of  report;  working  with  DT&E 
S October  6:  SE  Forum  -  Status  update  to  Component  Leaders 

•  November  17:  SE  Forum  -  Status  update  to  Component  Leaders 

•  November  22:  Components’  Reports  submitted  to  OSD/SE  (and  OSD/DT&E) 
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WSARA  Report 
Longer-Term  Schedule 


•  November  22,  2009:  Services  provide  baseline  organization  assessments 
and  plans  to  OSD  /  SE  and  DT&E 

•  November  2009  -  January  2010:  Development  of  Draft  Joint  Report 

-D/DT&E  and  D/SE  review  Service  assessments  for  Congressional 
Report 

-Components  provide  15  min  briefs  on  the  submitted  reports  (Target: 

December  11,  2009) 

-OSD  Writers  Draft  Sections  2-8 

-WSARA  DT&E/SE  JAT  Completes  Initial  Draft  Report  Review 
(Target:  December  23,  2009) 

•  January  2010:  Draft  Congressional  Report 

-  SE  Forum  updates  on  WSARA  report  status  in  January  /  February  /  March 

•  March  2010:  Submit  Congressional  Report 

•  Post-March  2010:  Obtain  Congressional  feedback  and  initiate  plans  for 
2011  report  accordingly 
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DoD  Systems  Engineering 

Mr.  Stephen  Welby 
Director,  Systems  Engineering 
Office  of  the  Director,  Defense  Research  and  Engineering 

12th  Annual  NDIA  Systems  Engineering  Conference 

October  27,  2009 
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Support  from  the  Top  for  Change 


Weapon  Systems  Acquisition  Reform  Act  of  2009 


•  Establishes  Director,  Systems  Engineering  (D, 
SE)  and  Director,  Developmental  Test  and 
Evaluation  (D,  DT&E)  as  principal  advisors  to 
the  SECDEF  and  the  USD(AT&L) 

•  Mandates  documented  assessment  of 
technological  maturity  and  integration  risk  of 
critical  technologies  for  MDAPs  during  the 
Technology  Development  (TD)  phase 

•  Establishes  D,  DT&E  and  D,  SE  joint  tracking 
and  Congressional  reporting  on  MDAP 
achievement  of  measurable  performance 
criteria 

•  Mandates  competitive  prototyping  and  MDA 
completion  of  a  formal  Post-Preliminary  Design 
Review  Assessment  for  all  MDAPs  before  MS  B; 
additional  MDA  certification  to  both  at  MS  B 

•  Strengthens  technical  analysis  of  cost  and 
schedule  breaches  during  the  Technology 
Development  (pre-MS  B)  and  the  Engineering 
and  Manufacturing  Development  (post-MS  B) 
phases 


President  Barack  Obama  hands  a  pen  to  U.S.  Rep.  Robert 
Andrews  (D-NJ)  as  he  signs  the  Weapons  Systems  Acquisition 
Reform  Act  in  the  Rose  Garden  at  the  White  House  Friday,  May 
22,  2009.  Standing  from  left  are:  Andrews,  Rep.  John  McHugh  (R- 
NY),  Sen.  Carl  Levin  (D-MI),  Rep.  Ike  Skelton  (D-MO)  and  Rep. 
Mike  Conaway  (R-TX).  Official  White  House  Photo  by  Samantha 
Appleton 
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MDAP-  Major  Defense  Acquisition  Program  (USC  2430) 
MDA  -  Milestone  Decision  Authority 


Director,  Systems  Engineering 


Rff  Director,  Systems  Engineering 

LV  >  Steve  Welby 

|  Terry  Jaggers,  Principal  Deputy 


System  Complexity  Analysis 
Red  Teaming 

Modeling  &  Simulation  Coordination 
Office 

Development  Planning 

SE  for  Systems  of  Systems 

Program  Protection/Acquisition 
Cyber  Security 

SE  Research  Center 


—  Program  Support  Reviews 

—  Systems  Engineering  Plans 

—  Program  Technical  Auditing 

—  OIPT/DAB/DSAB  Support 

—  DAES  Database  Analysis  and  Support 

—  Performance  Measurement 

—  Systemic  Root  Cause  Analysis 


—  Systems  and  Software  Engineering 

Policy,  Guidance,  Standards 

—  System  Safety 

—  Reliability,  Availability,  Maintainability 

—  Quality,  Manufacturing,  Producibility 

—  Human  Systems  Integration  (HSI) 

—  Technical  Workforce  Development 

—  Organizational  Capability  Assessment 

(WSARA) 


Responsible  to  provide  technical  support,  systems  engineering  oversight,  program 
development  and  mission  assurance  certification  to  USD(AT&L)  in  support  of  planned 

and  ongoing  acquisition  programs 
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Scope  of  DDR&E  Acquisition 
Program  Oversight  Efforts* 


Program 

Category 

Increasing 

cost/risk 

#  of 
Progs 

AC  AT  ID** 

$$$ 

MDA  =  AT&L 

93 

ACAT  1C** 

$$$ 

MDA  =  CAE 

r  52 

Special 

Interest** 

Any  $s 
Risk 

19 

MAIS,  ACAT 
IA 

$-$$$,  AIS 

30 

Pre-MDAP 

$$$ 

pre-MS  B 

53 

Pre-MAIS 

$-$$$,  AIS 
pre-MS  B 

10 

ACAT  II 

$$  <  ACAT  1 

8 

ACAT  III 

$  <  ACAT  II 

9 

Total 

274 

*Based  on  2009  T&E  Oversight  List  (Jan  5,  2009) 
**Major  Defense  Acquisition  Program  (MDAP) 
+Major  Automated  Information  System  (MAIS) 


%  Distribution  of  MDAPs 
by  Domain 


Other,  2% 


Fixed  Wing, 
20% 


Missiles,  8% 
Business,  2% 
Space  ,  6% 
Com  ms,  6% 


Land,  11% 

C2-ISR,  14% 


Rotary  Wing, 
15% 


Unmanned,  4% 

Ships,  9% 
Munitions,  3% 


MDA  -  Milestone  Decision  Authority 
TMA  -  Technology  Maturity  Assessment 
CAE  -  Component  Acquisition  Executive 
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Systems  Engineering 
Contributions  to  Acquisition 


•  Systems-level  technical  leadership 

•  Risk  identification  and  management 

•  Interface  management 

•  Life  cycle  focus 

•  Robust  exploration  of  the  need 

•  Achievable  system  design 

•  Integration  of  technical  disciplines 
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Systems  Engineering  Mission 


Execute  substantive  technical 
engagement  throughout  the  acquisition 
life  cycle  with  major  and  selected 
acquisition  efforts  across  DoD  to  apply 
best  Systems  Engineering  practices  to: 

-  Help  program  managers  identify  and 
mitigate  risks 

-  Shape  technical  planning  and  management 

-  Provide  insight  to  OSD  stakeholders 

-  Identify  systemic  issues  for  resolution 
above  the  program  level 
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DoD  5000.02  and  PL  111-23  - 
the  Changed  Acquisition  Landscape 


New  2366a  &  2366b  Certifications* 
MSA  /\  MSB 


MS  C 


CPD 

J 


CBA 

^ 


Materiel 

Solution 

Analysis 


Technology 

Development 


CDD 


Engineering  and 
Manufacturing 
Development 


Production  and 
Deployment 

_ 


O&S 


JCIDS  Process 


! 


PDR 


CDR 


Materiel 
Development 
Decision 
(MDD) 

Mandatory 

Competitive 

Prototypes 


t 


t 


PDR,  PDR 
Report  to  the 
MDA,  and 
Post-PDR- 
Assessment 
before  MS  B 


System -level 
CDR  with  an 
initial 
product 
baseline  and 
a  Post-CDR 
Report  to 
the  MDA 


Full  Rate  Production 
Decision  Review 


Renewed  emphasis  on 
manufacturing  across 
the  lifecycle 


Post-CDR 
Assessment  by 
the  MDA 
between  EMD 
sub-  phases 


“Knowledge-based”  Decision  Making  . .  .making  acquisition 
decisions  when  you  have  solid  evidence  and  acceptable  risk 
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*  Director,  SE  supports  MDA  certifications  including 

PDR  Report  assessment  at  MS  B 


New  Emphasis  on  Development  Planning 
and  Early  Systems  Engineering 


Development  Planning 


Development 


CBA:  Capabilities  Based  Assessment 
CDD:  Capability  Development  Document 
CDR:  Critical  Design  Review 
CPD:  Capability  Production  Document 
DP:  Development  Planning 
FRP  DR:  Full-Rate  Production  Decision 
Review 

ICD:  Initial  Capabilities  Document 
MDD:  Materiel  Development  Decision 
O&S:  Operations  and  Support 
PDR:  Preliminary  Design  Review 
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The  Current  Systems  Engineering 

Environment 


Systems  Engineers  confront  a  spectrum  of  issues  that  challenge 

“traditional”  systems  engineering 
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DDR&E  Imperatives 


1.  Accelerate  delivery  of  technical 
capabilities  to  win  the  current  fight 

2.  Prepare  for  an  uncertain  future 

3.  Reduce  the  cost,  acquisition  time  and  risk 
of  our  major  defense  acquisition 
programs 

4.  Develop  world  class  science,  technology, 
engineering,  and  mathematics  capabilities 
for  the  DoD  and  the  Nation 
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(Draft)  FY2010 

Systems  Engineering  Initiatives 


DDR&E  1  Accelerate  delivery  of  technical  capabilities  to  win  the  current  fight 

SE  1.1  Leverage  “lighter-weight”  tailored  Systems  Engineering  process  for 
urgent  needs,  rapid  fielding  and  technology  insertion 

Director,  Systems  Engineering  Focus: 

Support  the  current  fight,  manage  risk  with  discipline 


DDR&E  2  Prepare  for  an  uncertain  future 

SE  2.1  Develop  Systems  Engineering  techniques  to  formally  specify  and 
measure  adaptability/flexibility/adjustability  of  defense  systems  to 
operate  in  new  and  unknown  environments/missions 

SE  2.2  Develop  new  approaches  to  address  emerging  Systems  Engineering 
competencies  in  complex  systems,  large  scale  software,  and  trusted  and 
secured  systems 

SE  2.3  Conduct  Systems  of  Systems  analysis  in  support  of  system  and 
architecture  level  assessment  of  emerging  capabilities 

Director,  Systems  Engineering  Focus: 

Grow  engineering  capabilities  to  address  emerging  challenges 
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(Draft)  FY2010 

Systems  Engineering  Initiatives 


DDR&E  3  Reduce  the  cost,  acquisition  time  and  risk  of  our  Major  Defense 
Acquisition  Programs 

SE  3.1  Engage  continuously  with  Service  acquisition  efforts  -  provide 
mentorship  and  support  to  program  offices 

SE  3.2  Support  early  development  planning  for  emerging  acquisition  efforts 
per  WSARA 

SE  3.3  Review  and  approve  Systems  Engineering  Plans  for  all  MDAP  and 
MAIS  efforts  and  report  to  Congress 

SE  3.4  Eliminate  serial  oversight  -  Integrate  Systems  Engineering  Program 
Support  Reviews  with  specialty  reviews  across  DDR&E  and  A&T 

SE  3.5  Leverage  the  Systems  Engineering  process  for  major  systems 

acquisition  to  identify  and  mitigate  technical  and  programmatic  risks  early 

SE  3.6  Manage  risk  escapes  through  the  use  of  formal  DDR&E  red  teams  to 
provide  comprehensive  technical  assessment  of  critical  programs 

SE  3.7  Manage  system  vulnerability  and  mitigate  security  risk  through 
program  threat  protection  policy  and  assessment 

Director,  Systems  Engineering  Focus: 

Champion  Systems  Engineering  as  a  tool  to  improve  acquisition  quality 
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(Draft)  FY2010 
Systems  Engineering  Ir 


DDR&E  4  Develop  World  Class  Science,  Technology,  Engineering  and 
Mathematics  capabilities  for  the  DoD  and  the  Nation 

SE  4.1  Create  opportunities  to  attract,  foster  and  grow  future  DoD 
engineering  leaders 

SE  4.2  Engage  with  industry  to  develop  and  share  Systems  Engineering 
“Best  Practices” 


SE  4.3  Support  workforce  development,  competency  modeling  and 
assessment  and  certification  standards 


SE  4.4  Assess  Service  Systems  Engineering  capabilities  and  report  to 
Congress  per  WSARA 

SE  4.5  Develop,  support  and  coordinate  next  generation  Modeling, 
Simulation  and  Analysis  capabilities 

SE  4.6  Provide  consistent  Systems  Engineering  guidance  and  policy  to 
the  Services,  Agencies  and  industry 

Director,  Systems  Engineering  Focus: 

Develop  future  technical  leaders  across  the  acquisition  enterprise 
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Multi-Level  Engagement 


SE 

Policy  &  Guidance 

•  Systems  Engineering 

•  Software  Engineering 

Program  Support 

•  Program  Support  Reviews 

•  OIPT  and  SE  WIPTs 

•  AOTR,  Post-PDR/CDR  Review 
&  Assessment 

Workforce  Planning 

•  Competency  Models 

•  Certification  Requirements 

•  Education  &  Training 

Emerging  Concepts 

•  Systems  of  Systems 

•  SE  Research 

Outreach 

•  SE  Forum 

•  Engagement 
Strategy 


Congress  Statutory  Direction 


OSD 


Service  Acquisition 
Executives 


Policy  and  Guidance 


Requirement 

Developers 


ICD,  CDD,  CPD 


PEOs/Program  Offices 

Engineering  Centers  and 
Evaluation  Commands 

Prime  Contractors  and 
Supply  Chain 


Education  &  Collaboration  Infrastructure 
Professional/Industry  Associations 
DAU,  Academic  Institutions,  SERC,  International  Partners 


DAB,  ITAB,  DSAB,  OIPT,  PSR, 
SEP,  PPP,  Technical  Reviews, 
SE  WIPT 


Improved  SE  Methods, 
Processes,  and  Tools, 
International  and 
National  Standards 
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Systems  Engineering’s 
Partnership  with  Industry 


*  Systems  Engineering’s  industry  stakeholders 
include: 

-  Prime  and  sub  contractors 

-  Supply  chain  vendors 

-  Practicing  systems  engineers 

-  Systems  engineering  tool  vendors 

*  Systems  Engineering  leverages  industry  and 
professional  associations  to: 

-  Disseminate  policy  and  guidance 

-  Obtain  feedback  from  industry 

-  Promote  sound  systems  engineering  best  practices 
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Opportunities 


•  Acquisition  reform  efforts  have  recognized  criticality  of 
strong  Systems  Engineering  focus  for  program  success 

-  Systems  Engineering  toolkit  focused  on  identifying  and 
managing  risk  -  development  risk,  production  risk  and  life-cycle 

•  Growing  focus  on  addressing  “early-acquisition”  phases  - 
requirements  definition,  development  planning,  and  early 
acquisition 

-  Leading  to  more  informed  decisions  at  MS  B 

•  Our  development  processes  need  to  evolve  to  provide  faster 
product  cycles,  more  adaptable  products  and  address 
emerging  challenges 

•  Future  US  Defense  capabilities  depend  on  a  capable  US 
engineering  workforce  in  and  out  of  government 

-  Need  to  create  opportunities  to  grow  future  Engineering  Heroes 
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Systems  Engineering: 
Critical  to  Program  Success 


Innovation ,  Speed  and  Agility 
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